-- Logs begin at Thu 2022-06-23 10:11:02 UTC, end at Thu 2022-06-23 11:55:15 UTC. -- Jun 23 10:48:24.709704 dv-cinder-826607-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 10:48:24.719020 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 10:48:24 2022] *** Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: nodename: dv-cinder-826607-6-iscsi Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: machine: x86_64 Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: clock source: unix Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: pcre jit disabled Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: detected number of CPU cores: 4 Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: current working directory: / Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: detected binary path: /usr/bin/uwsgi-core Jun 23 10:48:24.721016 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: your processes number limit is 47680 Jun 23 10:48:24.721767 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: your memory page size is 4096 bytes Jun 23 10:48:24.721767 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: detected max file descriptor number: 2048 Jun 23 10:48:24.721767 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: lock engine: pthread robust mutexes Jun 23 10:48:24.721767 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: thunder lock: enabled Jun 23 10:48:24.721767 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 10:48:24.721767 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 10:48:24.767653 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: Python main interpreter initialized at 0x5643834ab9c0 Jun 23 10:48:24.767653 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: python threads support enabled Jun 23 10:48:24.767653 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: your server socket listen backlog is limited to 100 connections Jun 23 10:48:24.767653 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: your mercy for graceful operations on workers is 90 seconds Jun 23 10:48:24.768018 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 10:48:24.768097 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: *** Operational MODE: preforking *** Jun 23 10:48:24.768343 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: *** uWSGI is running in multiple interpreter mode *** Jun 23 10:48:24.768343 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: spawned uWSGI master process (pid: 80329) Jun 23 10:48:24.768543 dv-cinder-826607-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 10:48:24.771262 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: spawned uWSGI worker 1 (pid: 80342, cores: 1) Jun 23 10:48:24.771262 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: spawned uWSGI worker 2 (pid: 80343, cores: 1) Jun 23 10:48:24.771262 dv-cinder-826607-6-iscsi devstack@keystone.service[80329]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 10:48:30.246247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:30.246247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: from cryptography.utils import int_from_bytes Jun 23 10:48:30.247044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:30.247044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: from cryptography.utils import int_from_bytes Jun 23 10:48:30.247502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:30.247502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: from cryptography.utils import int_from_bytes Jun 23 10:48:30.247614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 10:48:30.247614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: from cryptography.utils import int_from_bytes Jun 23 10:48:30.295817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 10:48:30.295979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 10:48:30.296058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 10:48:30.296150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 10:48:30.296232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] command line args: [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 10:48:30.296346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] command line args: [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 10:48:30.296460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 10:48:30.296620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 10:48:30.296702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ================================================================================ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 10:48:30.296807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ================================================================================ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 10:48:30.296901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] admin_token = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] admin_token = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config_dir = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config_dir = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] config_source = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] debug = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] config_source = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.297832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] debug = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.298983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_config_append = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_config_append = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_dir = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.299972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_dir = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300789 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.300956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.301995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.302944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.303086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.303172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.303604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.303736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.303823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] publish_errors = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.303897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] publish_errors = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.304948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] standard_threads = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] standard_threads = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.305980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_journal = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_journal = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_json = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_json = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_stderr = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_stderr = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] use_syslog = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] use_syslog = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 10:48:30.306881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.306968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.307967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.308932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.309095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.309184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.309344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.309422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.309576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.309652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.309863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.310103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.310217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.310424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.310508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.310686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.310770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.310925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.311934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.312930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.313942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.314884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.315853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.316934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317173 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.317999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.318240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.318356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.318460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.318583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.318705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.318830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.318947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.319904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320726 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.320973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.321895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.322953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.323973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.324981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.325910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.326125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.326247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.326364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.326478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.326592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.326751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.326897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.327894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.328956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.329902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.330971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.331982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.external = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.external = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.332956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.password = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.password = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] auth.token = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] auth.token = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.333879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.334957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.335936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.336983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.337113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.337307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.337428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.337543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.337659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.337775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.337889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.338964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.339080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.339224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.339343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.339472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.339599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.339717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.339955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.340902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.341553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.341705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.341875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.342992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343367 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.343961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.344893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.345937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.346994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.347116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.347271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.347402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.347519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.347635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.348938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.349891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.350973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.351088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.351472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.357516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.357627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.357711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.357802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.357881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.357955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.358959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.359981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.360974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.361925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.362966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.363981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.364961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.365934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366079 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.366972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.367983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.368964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.369931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.370982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.371914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.372006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.372093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.372232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.372376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.372951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.373972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.374977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375477 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.375950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.377930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.378867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.379990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.380070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.380225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.380316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.380476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.380612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.380741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.382522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.382673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.382825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.382978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.383115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.383316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.383450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.383580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.383724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.383856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.383985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80343) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 10:48:30.384116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 10:48:30.384116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80342) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 10:48:30.412359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.412603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.412795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.413072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.413306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.413529 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.421886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.422811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.449098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.449331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.449846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.450080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.592220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.592459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.640672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.640903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.642038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.642274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.704742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.704966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.705186 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.705406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.705624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.705838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.706690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.706926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.707149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.707393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.707609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.707824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.731440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.731578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.731712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.731840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.731932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.732830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.733159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.743691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.743914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.756588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.756817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.757932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80343) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.758068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.758285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.758500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80342) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 10:48:30.775596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.775807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:30.776118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.776206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.776328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:30.776414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:30.776584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.776668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.776832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 10:48:30.776924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.777091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.777174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.777347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.777430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.777605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.777862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.788869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.789145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.789340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.789424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.789617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.789701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.789883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.790160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.797949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.798539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.801286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.801566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.801828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.801912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802385 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.802998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.803167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.803270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.803457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.803541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.803725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.803808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.803989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.804069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.804265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.804527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.830779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.831439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.832410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.832685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.833043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.833319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.837928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.838313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.838547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.838633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.838941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.839244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.860502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.860502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.860758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.860758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.860920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.860920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.861132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.861888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.873081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.873187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.873356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.873444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.877935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.878023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.881566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.881662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.881955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.882054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.886989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.887090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.891229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.891332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.894867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.894956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.895165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.895267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.895475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.895560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.895756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.895839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.896029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.896122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.896306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.896392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.915308 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.915595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.916271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.916360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.916438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.916513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.916597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.916597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.924440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.924708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.926414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.926688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.927316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.932480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.932569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.932569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.932569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.932569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.932569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.932979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.933090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.933372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.933662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.935876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.945747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.946031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.946319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.946598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.948294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.948591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.948889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.949034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.949314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.949601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.949879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.952198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.952494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.952791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.967527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.970680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.970914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.973962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.974216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.976008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.977532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.977903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.978181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.978470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.979261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.980880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.981153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.981457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.985975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.987611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.988818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.990398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:30.990806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.991091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.991501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.991778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.993630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.993917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.994309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:30.994585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.001417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:31.001703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.001978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.004795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:31.005082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.005361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.010957 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80343) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:31.011266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.011548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.011821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.012090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.012477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.012752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.013606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.013778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80342) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 10:48:31.013889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.014966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.015220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80343) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.015307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.015549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.015827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.016099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.016368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.016639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.016922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.017188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.017464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80342) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 10:48:31.052548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5643834ab9c0 pid: 80343 (default app) Jun 23 10:48:31.055164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5643834ab9c0 pid: 80342 (default app) Jun 23 10:48:31.055927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95006d23-79f4-4180-96f8-e6dd161e7c21 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:31.056231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95006d23-79f4-4180-96f8-e6dd161e7c21 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:31.056510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95006d23-79f4-4180-96f8-e6dd161e7c21 None None] PATH_INFO: `/v3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:31.061303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Thu Jun 23 10:48:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:39.700491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df454a-86fc-454d-99a2-ba7aeeb8fcd5 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:39.703274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df454a-86fc-454d-99a2-ba7aeeb8fcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:39.703274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df454a-86fc-454d-99a2-ba7aeeb8fcd5 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:39.705415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:48:39.710912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:39.711293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:39.711508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:39.793995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: INFO dbcounter [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] Registered counter for database keystone Jun 23 10:48:39.815478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG oslo_db.sqlalchemy.engines [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80343) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 10:48:39.821621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writer thread running {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 10:48:40.000438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG passlib.handlers.bcrypt [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80343) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 10:48:40.000778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG passlib.handlers.bcrypt [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80343) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 10:48:40.055665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:40.057260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:40.179420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 10:48:40.220959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df454a-86fc-454d-99a2-ba7aeeb8fcd5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:40.291434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: INFO dbcounter [None req-d1df454a-86fc-454d-99a2-ba7aeeb8fcd5 None None] Registered counter for database keystone Jun 23 10:48:40.310314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG oslo_db.sqlalchemy.engines [None req-d1df454a-86fc-454d-99a2-ba7aeeb8fcd5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80342) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 10:48:40.316496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writer thread running {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 10:48:40.436759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df454a-86fc-454d-99a2-ba7aeeb8fcd5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:40.441058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448ca0a-23fc-4310-8b72-d76d74855ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:40.441723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448ca0a-23fc-4310-8b72-d76d74855ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:40.442013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448ca0a-23fc-4310-8b72-d76d74855ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:40.442294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448ca0a-23fc-4310-8b72-d76d74855ba8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:40.450624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448ca0a-23fc-4310-8b72-d76d74855ba8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:40.503193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2448ca0a-23fc-4310-8b72-d76d74855ba8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 10:48:40.503539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448ca0a-23fc-4310-8b72-d76d74855ba8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:40.513427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Thu Jun 23 10:48:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 10:48:40.519474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:40.534312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b84bc-98b2-4641-bf80-84ae327cd366 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:40.538659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f986e94-e6ff-4d23-9f52-413ee31005c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:40.539869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986e94-e6ff-4d23-9f52-413ee31005c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:40.540157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986e94-e6ff-4d23-9f52-413ee31005c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:40.540433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f986e94-e6ff-4d23-9f52-413ee31005c1 admin admin] PATH_INFO: `/v3/projects/d945ea14d2824c65a22104765fba4b15` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:40.545160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f986e94-e6ff-4d23-9f52-413ee31005c1 admin admin] RBAC: Authorizing `identity:get_project(project_id=d945ea14d2824c65a22104765fba4b15)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:40.595475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7f986e94-e6ff-4d23-9f52-413ee31005c1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 10:48:40.595816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f986e94-e6ff-4d23-9f52-413ee31005c1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:40.600400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Thu Jun 23 10:48:40 2022] GET /identity/v3/projects/d945ea14d2824c65a22104765fba4b15 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:42.602055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0b300-8e41-4945-873d-7c51db08a8de None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:42.602598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0b300-8e41-4945-873d-7c51db08a8de None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:42.602598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0b300-8e41-4945-873d-7c51db08a8de None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:42.606858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:42.612351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc681e06-6bd3-4a2f-93a3-b5267a2b1901 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:42.612737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc681e06-6bd3-4a2f-93a3-b5267a2b1901 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:42.612953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc681e06-6bd3-4a2f-93a3-b5267a2b1901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:42.631537 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-dc681e06-6bd3-4a2f-93a3-b5267a2b1901 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:42.633070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dc681e06-6bd3-4a2f-93a3-b5267a2b1901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:42.641658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 10:48:42.664413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0b300-8e41-4945-873d-7c51db08a8de None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:42.676348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0b300-8e41-4945-873d-7c51db08a8de None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:42.680506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5d20a-560c-45ad-aedf-667dfb89d8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:42.681139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d20a-560c-45ad-aedf-667dfb89d8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:42.681431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d20a-560c-45ad-aedf-667dfb89d8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:42.681714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d20a-560c-45ad-aedf-667dfb89d8c5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:42.690091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5d20a-560c-45ad-aedf-667dfb89d8c5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:42.691912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5d20a-560c-45ad-aedf-667dfb89d8c5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:42.701244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Thu Jun 23 10:48:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 10:48:42.706790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc681e06-6bd3-4a2f-93a3-b5267a2b1901 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:42.716605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc681e06-6bd3-4a2f-93a3-b5267a2b1901 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:42.721205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab69c5-6e17-4c00-b61f-5e35a27f413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:42.722069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab69c5-6e17-4c00-b61f-5e35a27f413e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:42.722361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab69c5-6e17-4c00-b61f-5e35a27f413e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:42.722641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab69c5-6e17-4c00-b61f-5e35a27f413e admin admin] PATH_INFO: `/v3/users/912834523da04e79b38964af43312d61` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:42.724051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab69c5-6e17-4c00-b61f-5e35a27f413e admin admin] RBAC: Authorizing `identity:get_user(user_id=912834523da04e79b38964af43312d61)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:42.726153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab69c5-6e17-4c00-b61f-5e35a27f413e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:42.727759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Thu Jun 23 10:48:42 2022] GET /identity/v3/users/912834523da04e79b38964af43312d61 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:44.783581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799d087-cb18-44e8-a262-1b4670cb96dd None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.784460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799d087-cb18-44e8-a262-1b4670cb96dd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.784751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799d087-cb18-44e8-a262-1b4670cb96dd None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.786158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:44.791326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23a3c-c45b-469d-a48e-c2bf1c5f2a66 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.791727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23a3c-c45b-469d-a48e-c2bf1c5f2a66 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.791892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c23a3c-c45b-469d-a48e-c2bf1c5f2a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.808853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e1c23a3c-c45b-469d-a48e-c2bf1c5f2a66 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:44.810616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e1c23a3c-c45b-469d-a48e-c2bf1c5f2a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:44.845799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9c03a-9184-4f7a-b02a-364ceab6a54e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.846098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9c03a-9184-4f7a-b02a-364ceab6a54e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.846380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9c03a-9184-4f7a-b02a-364ceab6a54e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.847764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 6/11] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:48:44.853161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:44.853635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:44.853924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:44.892903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 6/12] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:44.894054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG passlib.handlers.bcrypt [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80342) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 10:48:44.894486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG passlib.handlers.bcrypt [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80342) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 10:48:44.916141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c23a3c-c45b-469d-a48e-c2bf1c5f2a66 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:44.940681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:44.942632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:44.944270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:45.022345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c23a3c-c45b-469d-a48e-c2bf1c5f2a66 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.042906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 7/13] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:45.055848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.056648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.056933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.057208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.065915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.068793 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:45.080214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:45.081074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:48:45.081938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 7/14] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:48:45.087144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.167827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb530a-7726-46a9-9d85-e13e5a0adbd9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.168811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.170221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.188615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb242c02-1011-4ab9-b5d8-b98d27a2586d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.204451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.205393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.205685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.205959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.222825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.223625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.223916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.224198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.224957 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:45.225271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.226943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.239093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 8/15] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:45.244515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.274057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.275610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.344146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e67ce9-ae64-4cc2-8915-08f5bc8f4b91 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.373715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 8/16] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:48:45.378633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.379459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.379565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.379816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.380093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] PATH_INFO: `/v3/users/912834523da04e79b38964af43312d61` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.381292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] RBAC: Authorizing `identity:get_user(user_id=912834523da04e79b38964af43312d61)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.382157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.382757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.384400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 9/17] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/users/912834523da04e79b38964af43312d61 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:45.390348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.405484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.482004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc67eed-a371-44c7-b967-e511701b33a3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.495072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f833a-b192-423e-8c1d-88a5df00e664 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.517296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.518095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.518427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.518664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.529197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.529985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.530270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.530545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.531810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.533291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.534912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 10/18] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:45.538132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.539740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.540523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:48:45.560215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:45.561052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:45.561858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:48:45.561858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:48:45.561858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:45.561858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:48:45.561858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 9/19] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:48:45.567042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:45.622866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.623743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:45.643306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9bd10-88d6-45ee-a958-f65e50d7280e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.667901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b754f42-a4c8-4904-b19f-3eda5c0abb0b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:45.677266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7067c-6011-4c3e-bf49-912138dd9a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.677924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7067c-6011-4c3e-bf49-912138dd9a52 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.678211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7067c-6011-4c3e-bf49-912138dd9a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.678498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7067c-6011-4c3e-bf49-912138dd9a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.679371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7067c-6011-4c3e-bf49-912138dd9a52 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.domain.id=default, user.id=912834523da04e79b38964af43312d61) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:45.679687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7067c-6011-4c3e-bf49-912138dd9a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.681131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7067c-6011-4c3e-bf49-912138dd9a52 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.688004 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 11/20] 10.222.0.43 () {56 vars in 1322 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/role_assignments?user.id=912834523da04e79b38964af43312d61&scope.domain.id=default&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:45.711612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508e8ba-0b72-4e21-8e16-7fedc123b526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:45.712297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e8ba-0b72-4e21-8e16-7fedc123b526 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:45.712585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e8ba-0b72-4e21-8e16-7fedc123b526 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:45.712870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e8ba-0b72-4e21-8e16-7fedc123b526 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:45.713622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e8ba-0b72-4e21-8e16-7fedc123b526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:45.713932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e8ba-0b72-4e21-8e16-7fedc123b526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:45.716063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508e8ba-0b72-4e21-8e16-7fedc123b526 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:45.735898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 10/21] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:48:47.845745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07cc38-a756-4c95-b20f-40a8824afc97 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.846764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07cc38-a756-4c95-b20f-40a8824afc97 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.847297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07cc38-a756-4c95-b20f-40a8824afc97 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.849149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 12/22] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:47.855462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fc91b-b5de-4942-a6bc-1711adac25e7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:47.856547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fc91b-b5de-4942-a6bc-1711adac25e7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:47.857120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fc91b-b5de-4942-a6bc-1711adac25e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:47.884225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1a2fc91b-b5de-4942-a6bc-1711adac25e7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:47.885783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1a2fc91b-b5de-4942-a6bc-1711adac25e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:47.987086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 11/23] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:48.024767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07cc38-a756-4c95-b20f-40a8824afc97 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:48.172013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07cc38-a756-4c95-b20f-40a8824afc97 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:48.230142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:48.231812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.232288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.232747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.251123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:48.259158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:48.260043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:48.260850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:48:48.264155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 13/24] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:48.272327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fc91b-b5de-4942-a6bc-1711adac25e7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:48.347327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:48.414941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fc91b-b5de-4942-a6bc-1711adac25e7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:48.459941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53138c-2a61-4d1f-8343-b970fe84c273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:48.460541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53138c-2a61-4d1f-8343-b970fe84c273 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.460829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53138c-2a61-4d1f-8343-b970fe84c273 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.461109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53138c-2a61-4d1f-8343-b970fe84c273 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.461868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53138c-2a61-4d1f-8343-b970fe84c273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:48.462180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53138c-2a61-4d1f-8343-b970fe84c273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:48.463595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53138c-2a61-4d1f-8343-b970fe84c273 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:48.489381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 12/25] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:48.495211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:48.498781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e17093-0c13-4bf5-bdcf-c6bdd2ca2fb8 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.499075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e17093-0c13-4bf5-bdcf-c6bdd2ca2fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.499386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e17093-0c13-4bf5-bdcf-c6bdd2ca2fb8 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.500730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 13/26] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:48:48.505311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:48.523555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0782f-12a0-4003-8477-b114f82e9207 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.523819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0782f-12a0-4003-8477-b114f82e9207 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.524073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0782f-12a0-4003-8477-b114f82e9207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.560198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a2c0782f-12a0-4003-8477-b114f82e9207 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:48.561414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a2c0782f-12a0-4003-8477-b114f82e9207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:48.696920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 14/27] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:48.698766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b216d-88b9-47aa-af2d-eb1ec56483e6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.699423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b216d-88b9-47aa-af2d-eb1ec56483e6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.702336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b216d-88b9-47aa-af2d-eb1ec56483e6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.704095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 15/28] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:48.717507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025adc4b-20e3-40d2-90a6-e63e3b222b5f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.718090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025adc4b-20e3-40d2-90a6-e63e3b222b5f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.718589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025adc4b-20e3-40d2-90a6-e63e3b222b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.721482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e481c-630c-427a-8b5b-6eb913648506 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:48.746423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-025adc4b-20e3-40d2-90a6-e63e3b222b5f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:48.747501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-025adc4b-20e3-40d2-90a6-e63e3b222b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:48.776267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbe9c6-82d6-4e4d-8e4b-a9ab15f3e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:48.777570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbe9c6-82d6-4e4d-8e4b-a9ab15f3e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.779224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:48.780085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbe9c6-82d6-4e4d-8e4b-a9ab15f3e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.780728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbe9c6-82d6-4e4d-8e4b-a9ab15f3e3b7 None admin] PATH_INFO: `/v3/users/912834523da04e79b38964af43312d61` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.782297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbe9c6-82d6-4e4d-8e4b-a9ab15f3e3b7 None admin] RBAC: Authorizing `identity:get_user(user_id=912834523da04e79b38964af43312d61)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:48.783967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbe9c6-82d6-4e4d-8e4b-a9ab15f3e3b7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:48.785905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 14/29] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:48 2022] GET /identity/v3/users/912834523da04e79b38964af43312d61 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:48.787458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa48359-a1c7-4b63-a63d-3b1e3d2ec638 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.787942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa48359-a1c7-4b63-a63d-3b1e3d2ec638 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.788467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa48359-a1c7-4b63-a63d-3b1e3d2ec638 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:48.789943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 15/30] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:48.790877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa48359-a1c7-4b63-a63d-3b1e3d2ec638 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:48.836152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 16/31] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:48.843200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025adc4b-20e3-40d2-90a6-e63e3b222b5f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:48.897510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:48.947687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa48359-a1c7-4b63-a63d-3b1e3d2ec638 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:48.978116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025adc4b-20e3-40d2-90a6-e63e3b222b5f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:48.993220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:48.994265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:48.994565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:48.994849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.008890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.011019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:49.019894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25586ca7-e5f1-4b8c-929c-2dc45736ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.020705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25586ca7-e5f1-4b8c-929c-2dc45736ec86 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.020999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25586ca7-e5f1-4b8c-929c-2dc45736ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.021277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25586ca7-e5f1-4b8c-929c-2dc45736ec86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.023400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25586ca7-e5f1-4b8c-929c-2dc45736ec86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.024883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25586ca7-e5f1-4b8c-929c-2dc45736ec86 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:49.026811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 17/32] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:49.029204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868d68d-cab1-4cf9-b2da-ea438333e897 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.029505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868d68d-cab1-4cf9-b2da-ea438333e897 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.029791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868d68d-cab1-4cf9-b2da-ea438333e897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:48:49.043441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:49.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:49.045091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:48:49.045091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:48:49.045091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:48:49.045091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:48:49.047955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 16/33] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:49.049003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.056439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9868d68d-cab1-4cf9-b2da-ea438333e897 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:49.057602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9868d68d-cab1-4cf9-b2da-ea438333e897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:49.095744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:49.148110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 18/34] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:49.150204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868d68d-cab1-4cf9-b2da-ea438333e897 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.178161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f83ee9-2a02-4a24-ac83-e6d27247a5bf None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.206192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:49.219635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.220493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.220925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.221424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.222371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.223949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:49.274218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868d68d-cab1-4cf9-b2da-ea438333e897 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.311389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.312540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.312975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.313400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] PATH_INFO: `/v3/domains/default/users/912834523da04e79b38964af43312d61/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.315806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=912834523da04e79b38964af43312d61, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.317419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:49.442032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 17/35] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 10:48:49 2022] POST /identity/v3/roles => generated 225 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:49.443011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.531699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 19/36] 10.222.0.43 () {58 vars in 1361 bytes} [Thu Jun 23 10:48:49 2022] PUT /identity/v3/domains/default/users/912834523da04e79b38964af43312d61/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:48:49.533849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.569532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:49.685973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d15466-6330-44a4-8f69-32def76561d2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.703319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c14d7-8fb7-4574-bf10-367476591568 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.734867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5ad29-d3be-4ac1-9519-8957a167e4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.735612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ad29-d3be-4ac1-9519-8957a167e4af None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.736187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ad29-d3be-4ac1-9519-8957a167e4af None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.736473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5ad29-d3be-4ac1-9519-8957a167e4af None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.737231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5ad29-d3be-4ac1-9519-8957a167e4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:49.737733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5ad29-d3be-4ac1-9519-8957a167e4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.740127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5ad29-d3be-4ac1-9519-8957a167e4af None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:49.754601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49311c-b642-434e-9c1a-c5aae44db100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.755654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49311c-b642-434e-9c1a-c5aae44db100 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.756118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49311c-b642-434e-9c1a-c5aae44db100 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.756568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49311c-b642-434e-9c1a-c5aae44db100 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.757512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49311c-b642-434e-9c1a-c5aae44db100 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.759325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49311c-b642-434e-9c1a-c5aae44db100 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:49.761954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 18/37] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:49.763643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff16a-b117-4bc5-a8a8-3b4629c3890e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.764010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff16a-b117-4bc5-a8a8-3b4629c3890e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.764365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ff16a-b117-4bc5-a8a8-3b4629c3890e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.765891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 19/38] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:49.769867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440810f-4aa6-4bb9-af12-0e62df0b85eb None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.769867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440810f-4aa6-4bb9-af12-0e62df0b85eb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.769867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440810f-4aa6-4bb9-af12-0e62df0b85eb None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.770441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 20/39] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:49.775118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440810f-4aa6-4bb9-af12-0e62df0b85eb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:49.827527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:49.863261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 20/40] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 10:48:49 2022] POST /identity/v3/roles => generated 219 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:49.864723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e509782-c517-4dc9-aebe-64207a6647f0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.865049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e509782-c517-4dc9-aebe-64207a6647f0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.865338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e509782-c517-4dc9-aebe-64207a6647f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.895627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2e509782-c517-4dc9-aebe-64207a6647f0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:49.898031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2e509782-c517-4dc9-aebe-64207a6647f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:49.910795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440810f-4aa6-4bb9-af12-0e62df0b85eb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:49.961423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:49.962409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:49.962875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:49.963355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:49.964233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:49.966372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.003940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 21/41] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:50.005919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549b63d-d412-411b-b394-2fd579227029 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.005919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549b63d-d412-411b-b394-2fd579227029 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.005919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549b63d-d412-411b-b394-2fd579227029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.027907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9549b63d-d412-411b-b394-2fd579227029 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:50.029831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9549b63d-d412-411b-b394-2fd579227029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:50.042375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.081326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 21/42] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:48:49 2022] POST /identity/v3/projects => generated 301 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:50.083962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.117763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 22/43] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:50.142900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549b63d-d412-411b-b394-2fd579227029 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.346169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:50.354064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eb077-52ae-4e8c-a610-cc8e21d642db None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.386505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549b63d-d412-411b-b394-2fd579227029 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.403124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.404008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.404301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.404583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.405798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.407354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.409263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 22/44] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:50.414681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:50.431947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e131f-c542-4a7b-8bdd-cf284914a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.432608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e131f-c542-4a7b-8bdd-cf284914a720 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.433096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e131f-c542-4a7b-8bdd-cf284914a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.433201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e131f-c542-4a7b-8bdd-cf284914a720 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.433983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e131f-c542-4a7b-8bdd-cf284914a720 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.435548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e131f-c542-4a7b-8bdd-cf284914a720 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.495138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 23/45] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 10:48:50 2022] POST /identity/v3/roles => generated 223 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:50.521718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd97d-2784-49f3-a7f8-a12184f4636f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:50.561055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685fb8c-eb6e-4f54-9000-44f11d234f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:50.561752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685fb8c-eb6e-4f54-9000-44f11d234f61 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:50.562044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685fb8c-eb6e-4f54-9000-44f11d234f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:50.562331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685fb8c-eb6e-4f54-9000-44f11d234f61 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:50.563068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685fb8c-eb6e-4f54-9000-44f11d234f61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:50.564598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685fb8c-eb6e-4f54-9000-44f11d234f61 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:50.657564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 23/46] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:48:50 2022] POST /identity/v3/projects => generated 312 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:48:51.693373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd58264-ecb6-4798-a08a-528b4a9ee3de None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.693889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd58264-ecb6-4798-a08a-528b4a9ee3de None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.693973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd58264-ecb6-4798-a08a-528b4a9ee3de None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.695376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 24/47] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:51.700947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92725-2cac-4404-b414-2c6e922a198e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.701350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92725-2cac-4404-b414-2c6e922a198e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.701508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92725-2cac-4404-b414-2c6e922a198e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.718743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-abc92725-2cac-4404-b414-2c6e922a198e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:51.720293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-abc92725-2cac-4404-b414-2c6e922a198e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:51.731304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.787267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 24/48] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:51.812951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd58264-ecb6-4798-a08a-528b4a9ee3de None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:51.822034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:51.920099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd58264-ecb6-4798-a08a-528b4a9ee3de None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:51.953888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:51.954601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:51.954889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:51.955168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:51.966805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:51.975930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:51.976943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:48:51.977770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 25/49] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:51.982984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc92725-2cac-4404-b414-2c6e922a198e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.079906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc92725-2cac-4404-b414-2c6e922a198e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.113993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.114658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.114944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.115250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.115999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:52.116308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.117692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.125797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 25/50] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.131233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.195000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.229767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ef35-c15e-4b5b-9a3d-4e43ef2f7750 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.261782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.263207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.263488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.263781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] PATH_INFO: `/v3/users/912834523da04e79b38964af43312d61` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.264961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] RBAC: Authorizing `identity:get_user(user_id=912834523da04e79b38964af43312d61)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.266413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.268061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 26/51] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/users/912834523da04e79b38964af43312d61 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.273401 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.285263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.373065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165590f-9bd4-4227-a369-3e66a9719af2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.408077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d75fc-6729-4986-bef3-908c21085686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.409070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d75fc-6729-4986-bef3-908c21085686 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.409367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d75fc-6729-4986-bef3-908c21085686 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.409653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d75fc-6729-4986-bef3-908c21085686 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.410839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d75fc-6729-4986-bef3-908c21085686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.412348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d75fc-6729-4986-bef3-908c21085686 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.413963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 26/52] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.420009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.519879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecce1-3d9a-4cd0-9f00-f1f686b7871c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.552974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2613e9-bc9f-49f9-bc1c-5f860b97ab79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:52.553637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2613e9-bc9f-49f9-bc1c-5f860b97ab79 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.553930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2613e9-bc9f-49f9-bc1c-5f860b97ab79 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.554213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2613e9-bc9f-49f9-bc1c-5f860b97ab79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.555045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2613e9-bc9f-49f9-bc1c-5f860b97ab79 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.domain.id=default, user.id=912834523da04e79b38964af43312d61) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:52.555396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2613e9-bc9f-49f9-bc1c-5f860b97ab79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:52.556845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2613e9-bc9f-49f9-bc1c-5f860b97ab79 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:52.559844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.565925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 27/53] 10.222.0.43 () {56 vars in 1322 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/role_assignments?user.id=912834523da04e79b38964af43312d61&scope.domain.id=default&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:52.751008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e80d6-e1f7-4126-b769-6d609ccd274d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.751528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e80d6-e1f7-4126-b769-6d609ccd274d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.751612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e80d6-e1f7-4126-b769-6d609ccd274d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.752978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 27/54] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:52.758222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18272f97-51a8-403c-95c6-5c515480295a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:52.758614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18272f97-51a8-403c-95c6-5c515480295a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:52.758795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18272f97-51a8-403c-95c6-5c515480295a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:52.775654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-18272f97-51a8-403c-95c6-5c515480295a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:52.776863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-18272f97-51a8-403c-95c6-5c515480295a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:52.841981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 28/55] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:52.866012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e80d6-e1f7-4126-b769-6d609ccd274d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:52.943065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:52.966926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e80d6-e1f7-4126-b769-6d609ccd274d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:52.999995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c9338-1d72-4ef5-888d-f05ed1693f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:53.000782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c9338-1d72-4ef5-888d-f05ed1693f55 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:53.001070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c9338-1d72-4ef5-888d-f05ed1693f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:53.001353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c9338-1d72-4ef5-888d-f05ed1693f55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:53.002528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c9338-1d72-4ef5-888d-f05ed1693f55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:53.004008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c9338-1d72-4ef5-888d-f05ed1693f55 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:53.005623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 28/56] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:53.011385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18272f97-51a8-403c-95c6-5c515480295a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:53.108269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18272f97-51a8-403c-95c6-5c515480295a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:53.133247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:53.141265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779750c-5e3f-4ab4-8206-c28bae78ab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:53.141930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779750c-5e3f-4ab4-8206-c28bae78ab9f None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:53.142223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779750c-5e3f-4ab4-8206-c28bae78ab9f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:53.142509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779750c-5e3f-4ab4-8206-c28bae78ab9f None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:53.143244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779750c-5e3f-4ab4-8206-c28bae78ab9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:53.144707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779750c-5e3f-4ab4-8206-c28bae78ab9f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:53.207267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 29/57] 10.222.0.43 () {60 vars in 1140 bytes} [Thu Jun 23 10:48:53 2022] POST /identity/v3/projects => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:55.229064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d8968-63ae-4d8f-ab06-f0a8bf96db07 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.229567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d8968-63ae-4d8f-ab06-f0a8bf96db07 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.229567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d8968-63ae-4d8f-ab06-f0a8bf96db07 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.230958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 29/58] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:55.236092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2fe66-8b8c-474c-a86c-ee2f40c1fb4c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.236502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2fe66-8b8c-474c-a86c-ee2f40c1fb4c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.236668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2fe66-8b8c-474c-a86c-ee2f40c1fb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.253326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-22e2fe66-8b8c-474c-a86c-ee2f40c1fb4c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:55.254509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-22e2fe66-8b8c-474c-a86c-ee2f40c1fb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:55.318863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 30/59] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:55.343328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d8968-63ae-4d8f-ab06-f0a8bf96db07 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:55.442072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d8968-63ae-4d8f-ab06-f0a8bf96db07 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:55.474257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d89a-4a0e-462b-be17-771308ab5c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:55.475037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d89a-4a0e-462b-be17-771308ab5c1c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.475616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d89a-4a0e-462b-be17-771308ab5c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.475901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d89a-4a0e-462b-be17-771308ab5c1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.477079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40d89a-4a0e-462b-be17-771308ab5c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:55.478532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40d89a-4a0e-462b-be17-771308ab5c1c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:55.480203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 30/60] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:48:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:55.486048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2fe66-8b8c-474c-a86c-ee2f40c1fb4c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:55.588244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2fe66-8b8c-474c-a86c-ee2f40c1fb4c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:55.599059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:55.622108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc077d87-c4c9-44cc-888b-ea9fb5457eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:55.622572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc077d87-c4c9-44cc-888b-ea9fb5457eb7 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:55.622860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc077d87-c4c9-44cc-888b-ea9fb5457eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:55.623137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc077d87-c4c9-44cc-888b-ea9fb5457eb7 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:55.623884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc077d87-c4c9-44cc-888b-ea9fb5457eb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:55.625333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc077d87-c4c9-44cc-888b-ea9fb5457eb7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:55.711563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 31/61] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:48:55 2022] POST /identity/v3/users => generated 275 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:48:58.825577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12922aa5-588f-4520-9b93-bb9f5208a3dd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.826584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12922aa5-588f-4520-9b93-bb9f5208a3dd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.827025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12922aa5-588f-4520-9b93-bb9f5208a3dd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.828589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 31/62] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:58.833887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53c5ca-9ce6-4353-8375-cd1cc1a0293d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.834689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53c5ca-9ce6-4353-8375-cd1cc1a0293d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.835137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53c5ca-9ce6-4353-8375-cd1cc1a0293d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.860205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4f53c5ca-9ce6-4353-8375-cd1cc1a0293d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:58.861671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4f53c5ca-9ce6-4353-8375-cd1cc1a0293d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:58.953890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 32/63] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:58.984041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00b32-0312-4265-8c72-f88b457cad4f None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:58.984356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00b32-0312-4265-8c72-f88b457cad4f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:58.984625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00b32-0312-4265-8c72-f88b457cad4f None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:58.986008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 32/64] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:58.989045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53c5ca-9ce6-4353-8375-cd1cc1a0293d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.002895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84976979-7980-4b15-8d29-2e3926f4e3f1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.003226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84976979-7980-4b15-8d29-2e3926f4e3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.003520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84976979-7980-4b15-8d29-2e3926f4e3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.024810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-84976979-7980-4b15-8d29-2e3926f4e3f1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:59.026018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-84976979-7980-4b15-8d29-2e3926f4e3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:59.041301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.121193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.128084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53c5ca-9ce6-4353-8375-cd1cc1a0293d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.171698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.172890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.173558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.174058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.191862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:59.204042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:59.204982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:48:59.205779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:48:59.208257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 33/65] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:59.209767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 33/66] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:59.216329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.234909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84976979-7980-4b15-8d29-2e3926f4e3f1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.365511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84976979-7980-4b15-8d29-2e3926f4e3f1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.365729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c2df-3f2e-4aef-8cd0-26f4cf971157 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.410359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.410562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c1837-5397-456e-8fbf-d08443759c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.410970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c1837-5397-456e-8fbf-d08443759c29 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.411304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c1837-5397-456e-8fbf-d08443759c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.411700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.411809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c1837-5397-456e-8fbf-d08443759c29 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.412220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.412517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c1837-5397-456e-8fbf-d08443759c29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:59.412655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.412984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c1837-5397-456e-8fbf-d08443759c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.414576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c1837-5397-456e-8fbf-d08443759c29 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.425808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:59.426102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 34/67] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.427739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6266973-5af3-48d4-ba0b-3c9cfee0c789 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.428199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6266973-5af3-48d4-ba0b-3c9cfee0c789 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.428637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6266973-5af3-48d4-ba0b-3c9cfee0c789 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.430043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 35/68] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:59.436083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1fe1-4c0f-4604-8701-c48480224aad None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.436561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1fe1-4c0f-4604-8701-c48480224aad None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.437003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1fe1-4c0f-4604-8701-c48480224aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:59.439824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:59.440900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:48:59.441789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 34/69] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:48:59.442636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.444925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.461087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-295c1fe1-4c0f-4604-8701-c48480224aad None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:59.462399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-295c1fe1-4c0f-4604-8701-c48480224aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:59.476004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.528637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 36/70] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:48:59.529491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1fe1-4c0f-4604-8701-c48480224aad None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.547448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd485e19-95d2-4f55-9612-c09d01728480 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.580611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ac7e8-dc69-4d53-858b-7e6df79c8c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.581483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ac7e8-dc69-4d53-858b-7e6df79c8c15 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.581768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ac7e8-dc69-4d53-858b-7e6df79c8c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.582048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ac7e8-dc69-4d53-858b-7e6df79c8c15 None admin] PATH_INFO: `/v3/users/912834523da04e79b38964af43312d61` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.583278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ac7e8-dc69-4d53-858b-7e6df79c8c15 None admin] RBAC: Authorizing `identity:get_user(user_id=912834523da04e79b38964af43312d61)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.584739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ac7e8-dc69-4d53-858b-7e6df79c8c15 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.586380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 35/71] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/users/912834523da04e79b38964af43312d61 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.587572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bed16-9829-4faa-86d4-f27ca0584910 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.587863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bed16-9829-4faa-86d4-f27ca0584910 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.588150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bed16-9829-4faa-86d4-f27ca0584910 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.589360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 36/72] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:59.590956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa6663-7652-4091-ae3a-55368831b9c9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.591262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa6663-7652-4091-ae3a-55368831b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.595323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa6663-7652-4091-ae3a-55368831b9c9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.595323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 37/73] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:48:59.595323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa6663-7652-4091-ae3a-55368831b9c9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.634844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1fe1-4c0f-4604-8701-c48480224aad None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.664927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.669154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.669828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.670120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.670407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.671159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:48:59.671505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.672911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.681268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 37/74] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:48:59.682058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.695787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.696958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa6663-7652-4091-ae3a-55368831b9c9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.732350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3574b-03f0-4539-8a69-8a6b0492d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.733171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3574b-03f0-4539-8a69-8a6b0492d692 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.733464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3574b-03f0-4539-8a69-8a6b0492d692 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.733747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3574b-03f0-4539-8a69-8a6b0492d692 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.743979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3574b-03f0-4539-8a69-8a6b0492d692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1fe3574b-03f0-4539-8a69-8a6b0492d692 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:48:59.752242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:48:59.753137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:48:59.754096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 38/75] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:48:59.756707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da27987-e700-4541-adf6-b14445c09db3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.757014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da27987-e700-4541-adf6-b14445c09db3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.757303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da27987-e700-4541-adf6-b14445c09db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.774323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0da27987-e700-4541-adf6-b14445c09db3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:59.775604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0da27987-e700-4541-adf6-b14445c09db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:59.784810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b17278-fcf8-47e9-85d6-1d78ad13567e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:48:59.819069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:48:59.819746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.820036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.820366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.841387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 39/76] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:59.842832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de2c92-267d-4a31-ae01-874fbf59d53b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:48:59.843130 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de2c92-267d-4a31-ae01-874fbf59d53b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:48:59.843477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de2c92-267d-4a31-ae01-874fbf59d53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:48:59.860539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d4de2c92-267d-4a31-ae01-874fbf59d53b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:48:59.861862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d4de2c92-267d-4a31-ae01-874fbf59d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:48:59.895063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:48:59.896592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:48:59.909060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.922096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:48:59.933105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 40/77] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:48:59.934915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de2c92-267d-4a31-ae01-874fbf59d53b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:48:59.998291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 38/78] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:48:59.999134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.033726 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de2c92-267d-4a31-ae01-874fbf59d53b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.067444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.068319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.068610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.068893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.082736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.084251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.085924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 41/79] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.086694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.097033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72484427-8dca-4aa1-9994-aff5851bc070 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.108066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.131863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.132543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.132838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.133127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.133885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:00.134204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.135266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.135649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.144446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 39/80] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:48:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.145754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.188051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75f7d7-a616-41ed-91b1-47972c5d247a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.222476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.223298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.223587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.223871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.225060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.226790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.228604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 42/81] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.229184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.249760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f03f71-d9c0-4bff-ac1c-fa27e4311628 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.282697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.283539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.283831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.284115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.294194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:00.302011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:00.302872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:00.303676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 40/82] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:00.304306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.334334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11f66f-fec6-4e35-bb5b-b66daf0b64b8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.338607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.352325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.369622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.370482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.370767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.371761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.373403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.375042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.376790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 43/83] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.377553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.408578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccbf2c-79a7-42bf-bddc-15a94846e6af None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.441619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.442203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.442496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.442775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.476106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c1c1c-1482-4878-86b3-b4f2f05ee9df None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.509000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.509366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.510343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.510493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.510753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.511096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.512918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.514418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.516160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 44/84] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.516890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.542078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.567682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.611526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 41/85] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:00.612371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.618998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1a013-7b71-4ddc-addb-446e2f14c339 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.653820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.654491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.654785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.655064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.655813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.657259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.711364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa9a38-7326-4eff-b7d1-62b9f5725a73 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.734634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 45/86] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:00 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:00.735657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.740746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.746679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.747614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.747930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.748229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.749031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:00.749355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.750827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.760839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 42/87] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.761733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:00.871788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644b400-3bf0-4269-97e7-a9c2ff076cb6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.878980 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7be5d-b87b-474c-8c4e-3af17ed28698 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:00.920910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:00.947017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.947560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:00.947829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.947943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:00.948016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.948340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.948431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:00.949680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:00.949789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=912834523da04e79b38964af43312d61) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:00.950120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:00.952190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:00.959169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 46/88] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/role_assignments?user.id=912834523da04e79b38964af43312d61&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:00.963728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.036598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.047128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.048687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.093698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef31ec1-7b9b-4671-a904-66740a38e52e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.138624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.139976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.140540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.140879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.142161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.144470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.146595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 47/89] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.147623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.189033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 43/90] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:01.190292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.205087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.298207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.305961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f58d6-87d5-43d8-9b3b-9ae677a36f57 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.313440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8a2a-8431-41d0-af38-0e019f6ee1ee None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.343689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.344652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.345611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.346082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.347311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=1f7e314d61674e539a24c3878d778b9a) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:01.347840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.349479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.357475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 44/91] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/role_assignments?user.id=1f7e314d61674e539a24c3878d778b9a&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.358635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.392326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.393223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.393570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.393856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.395126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.402859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.404738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 48/92] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.412241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.574334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5674806-cbf4-4637-b099-2238aa6e8603 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.601997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370c082-d61e-4460-b543-22cee4a1e3b5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.618301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.637944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.639200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.639754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.640259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.641706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.643428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.645424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 45/93] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:01.651025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:01.686013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:01.688187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.688863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.689155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.689437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.801518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b466a3b-b1cb-4edc-b51e-687868923afe None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:01.841348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:01.842931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:01.843613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:01.844274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:01.844974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.845572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] RBAC: Adding query filter params (role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=1f7e314d61674e539a24c3878d778b9a) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:01.846088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:01.846487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.848038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:01.855961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 46/94] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/role_assignments?user.id=1f7e314d61674e539a24c3878d778b9a&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:01.995094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 49/95] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:02.001362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:02.129334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:02.145223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbb58a-853d-4113-a2e7-15d668b9ec96 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:02.193107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c369c61-f37a-421a-bece-d3f72d5bfea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:02.193998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369c61-f37a-421a-bece-d3f72d5bfea2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.194288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369c61-f37a-421a-bece-d3f72d5bfea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.194576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c369c61-f37a-421a-bece-d3f72d5bfea2 None admin] PATH_INFO: `/v3/projects/10b68da7f8344ed7ba4f153f93683b8a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.195824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c369c61-f37a-421a-bece-d3f72d5bfea2 None admin] RBAC: Authorizing `identity:get_project(project_id=10b68da7f8344ed7ba4f153f93683b8a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:02.197297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c369c61-f37a-421a-bece-d3f72d5bfea2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:02.200071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 47/96] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:02 2022] GET /identity/v3/projects/10b68da7f8344ed7ba4f153f93683b8a => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:02.204767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:02.221167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:02.347734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fb378-5485-48f8-8abe-4d913ec48afb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:02.393120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69468054-038f-45d4-9217-b66da99c9c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:02.394154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468054-038f-45d4-9217-b66da99c9c0f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:02.394661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468054-038f-45d4-9217-b66da99c9c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:02.395141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69468054-038f-45d4-9217-b66da99c9c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:02.396208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69468054-038f-45d4-9217-b66da99c9c0f None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=10b68da7f8344ed7ba4f153f93683b8a, user.id=1f7e314d61674e539a24c3878d778b9a) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:02.396727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69468054-038f-45d4-9217-b66da99c9c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:02.398398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69468054-038f-45d4-9217-b66da99c9c0f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:02.406962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 50/97] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:02 2022] GET /identity/v3/role_assignments?user.id=1f7e314d61674e539a24c3878d778b9a&scope.project.id=10b68da7f8344ed7ba4f153f93683b8a&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:03.781055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771077-df32-401c-82a2-90ae45a4247e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.781569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771077-df32-401c-82a2-90ae45a4247e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.781644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771077-df32-401c-82a2-90ae45a4247e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.783155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 48/98] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:03.788797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327c640-d961-4485-9e73-925eb487c709 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.789636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327c640-d961-4485-9e73-925eb487c709 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.790121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327c640-d961-4485-9e73-925eb487c709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.808186 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2327c640-d961-4485-9e73-925eb487c709 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:03.809737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2327c640-d961-4485-9e73-925eb487c709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:03.893503 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 51/99] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:03.949094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54771077-df32-401c-82a2-90ae45a4247e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:03.993322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2a35b-b65a-4199-bcdb-4f2c9c6e0c5a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:03.993950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2a35b-b65a-4199-bcdb-4f2c9c6e0c5a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:03.994518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2a35b-b65a-4199-bcdb-4f2c9c6e0c5a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:03.996245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 52/100] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:04.002109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbae69-f581-4ae6-bfe7-4bf84fb76279 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.002637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbae69-f581-4ae6-bfe7-4bf84fb76279 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.003140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbae69-f581-4ae6-bfe7-4bf84fb76279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.008478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.035646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c3cbae69-f581-4ae6-bfe7-4bf84fb76279 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:04.038044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c3cbae69-f581-4ae6-bfe7-4bf84fb76279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:04.148940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54771077-df32-401c-82a2-90ae45a4247e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.191669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 53/101] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:04.216577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.217835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.218326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.218857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.222792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.224619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.226769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 49/102] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:04.232821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbae69-f581-4ae6-bfe7-4bf84fb76279 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.233128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.285092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.377313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbae69-f581-4ae6-bfe7-4bf84fb76279 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.411153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.417123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630336ea-aba0-4540-9eed-4bd5f038152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.418402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630336ea-aba0-4540-9eed-4bd5f038152e None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.418688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630336ea-aba0-4540-9eed-4bd5f038152e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.419253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630336ea-aba0-4540-9eed-4bd5f038152e None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.419989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630336ea-aba0-4540-9eed-4bd5f038152e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.426725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630336ea-aba0-4540-9eed-4bd5f038152e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.426860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c01268-9db5-4fec-87a2-d74d3c4cd3d1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.460053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a584-31ff-4de3-a117-dd616201a35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.460976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a584-31ff-4de3-a117-dd616201a35c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.461341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a584-31ff-4de3-a117-dd616201a35c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.461813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a584-31ff-4de3-a117-dd616201a35c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.472685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a584-31ff-4de3-a117-dd616201a35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-40a7a584-31ff-4de3-a117-dd616201a35c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:04.481356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:04.485590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 50/103] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:04.486353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020e6fa-0649-4633-93a6-8c8e90638724 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.487071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020e6fa-0649-4633-93a6-8c8e90638724 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.487071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020e6fa-0649-4633-93a6-8c8e90638724 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.487071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 51/104] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:04.487399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82cc3-e236-4e0b-964f-e19d605d128c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.487970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82cc3-e236-4e0b-964f-e19d605d128c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.487970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82cc3-e236-4e0b-964f-e19d605d128c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.490573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 52/105] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:04.494793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dba30-2edd-4490-8022-9e03db4d5caf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.495090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dba30-2edd-4490-8022-9e03db4d5caf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.495399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dba30-2edd-4490-8022-9e03db4d5caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.513443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b27dba30-2edd-4490-8022-9e03db4d5caf None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:04.514640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b27dba30-2edd-4490-8022-9e03db4d5caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:04.516287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 54/106] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:49:04 2022] POST /identity/v3/projects => generated 302 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:04.519556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425f73-f9b5-4054-a004-b545143b78d4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.520262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425f73-f9b5-4054-a004-b545143b78d4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.520743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425f73-f9b5-4054-a004-b545143b78d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.544635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8b425f73-f9b5-4054-a004-b545143b78d4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:04.546425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8b425f73-f9b5-4054-a004-b545143b78d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:04.632600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 53/107] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:04.633809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dba30-2edd-4490-8022-9e03db4d5caf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.643190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 55/108] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:04.644511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b831a-f64c-4d02-af46-bae16c2e5ea9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.644804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b831a-f64c-4d02-af46-bae16c2e5ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.645087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b831a-f64c-4d02-af46-bae16c2e5ea9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.646690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 56/109] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:04.654302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0e82-cfa4-409c-b12a-55386f46702e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.654613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0e82-cfa4-409c-b12a-55386f46702e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.654905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0e82-cfa4-409c-b12a-55386f46702e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.655883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.673955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-099c0e82-cfa4-409c-b12a-55386f46702e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:04.675748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-099c0e82-cfa4-409c-b12a-55386f46702e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:04.747911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dba30-2edd-4490-8022-9e03db4d5caf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.757659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 57/110] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:04.758523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0e82-cfa4-409c-b12a-55386f46702e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.797261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.799583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.799877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.800168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.801008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:04.801321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:04.802702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:04.810720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.816735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 54/111] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:04.817002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:04.885406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0e82-cfa4-409c-b12a-55386f46702e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.938345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:04.967075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5a543-5604-4e19-849c-6061f3c02322 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:04.972323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:04.973362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:04.973883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:04.974337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:04.988856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:04.999141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:05.000273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:05.001127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:05.003642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 58/112] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:05.005152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.009581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.011852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.012319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.012607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.026030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:05.036841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:05.037738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:05.038610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 55/113] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:05.039287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.123701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab0aea-7033-4dfb-b2df-a6a43dde32e1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.124328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.156905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393c899-ba97-4866-a446-d951238b1ef1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.160566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.160936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.161780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.162532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.175354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:05.184005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:05.184896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:05.185656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 59/114] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:05.186320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.200606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.201861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.202357 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.202950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] PATH_INFO: `/v3/users/f4052cebea2b4d608b8155c269d1308f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.212284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.220864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] RBAC: Authorizing `identity:get_user(user_id=f4052cebea2b4d608b8155c269d1308f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.222616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.224507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 56/115] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/users/f4052cebea2b4d608b8155c269d1308f => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:05.227377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.304570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4008c7-7f29-4b0e-b76d-cbd0b10dd018 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.336259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f33e94-f443-498f-8066-6cac0d51a556 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.347089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.347878 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.348242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.348879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.350296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:05.350681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.352358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.363485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 60/116] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:05.364318 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.377928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.378916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.379411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.380211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.381170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:05.381936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.383811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.395089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 57/117] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:05.396177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.405752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.468514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.505842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c252-74c7-4090-b659-400c4e197a1f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.517618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ef2df-e800-494e-815c-b38911f96d6e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.543499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.544182 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.544472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.544755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.545524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:05.545860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.547254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.555261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 61/118] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:05.556052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.557510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff5d10-f2d7-4303-96e1-b44582f5068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.558744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff5d10-f2d7-4303-96e1-b44582f5068d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.559411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff5d10-f2d7-4303-96e1-b44582f5068d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.559411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff5d10-f2d7-4303-96e1-b44582f5068d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.630593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff5d10-f2d7-4303-96e1-b44582f5068d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.632118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff5d10-f2d7-4303-96e1-b44582f5068d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.667323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.680181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bf96b-6d39-47c4-a03e-ed6ff213029e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.690602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.715750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.716637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.716933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.717219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.718568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.720008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.726357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 62/119] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:05.726357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.742651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 58/120] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:05.744096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d5077-eb0a-4a0a-9c38-dcb8f27e7bdf None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.744400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d5077-eb0a-4a0a-9c38-dcb8f27e7bdf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.744687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d5077-eb0a-4a0a-9c38-dcb8f27e7bdf None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.746084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 59/121] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:05.746801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d5077-eb0a-4a0a-9c38-dcb8f27e7bdf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.845131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9d562-b385-40ac-8a5b-5e22729a0b66 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.860114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d5077-eb0a-4a0a-9c38-dcb8f27e7bdf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:05.885601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.887008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.887336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.887622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] PATH_INFO: `/v3/users/912834523da04e79b38964af43312d61` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.888880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=912834523da04e79b38964af43312d61)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.890349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.892083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 63/122] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/users/912834523da04e79b38964af43312d61 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:05.892944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.904875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:05.906221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:05.906820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:05.907107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:05.908852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:05.911103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:05.919295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 60/123] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:05.919295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:05.920993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.954359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:05.998971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7fd52-a90e-4939-a033-190d224a7aa7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.030685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc192c75-b926-4042-bf46-58a87d4fef00 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.039126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05680a1e-9fcd-45a3-ae27-03ddc7669a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.040233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680a1e-9fcd-45a3-ae27-03ddc7669a13 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.040528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680a1e-9fcd-45a3-ae27-03ddc7669a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.040820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05680a1e-9fcd-45a3-ae27-03ddc7669a13 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.042011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05680a1e-9fcd-45a3-ae27-03ddc7669a13 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.043669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05680a1e-9fcd-45a3-ae27-03ddc7669a13 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.045545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 64/124] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.046685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c406403-1fd6-4283-916e-bc1d578f6d6f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.046986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c406403-1fd6-4283-916e-bc1d578f6d6f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.047297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c406403-1fd6-4283-916e-bc1d578f6d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.064594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8c406403-1fd6-4283-916e-bc1d578f6d6f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:06.065787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8c406403-1fd6-4283-916e-bc1d578f6d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:06.071444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.072317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.072607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.072889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.074712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.076288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.078002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 61/125] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:05 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.078763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.142436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 65/126] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:06.148092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c406403-1fd6-4283-916e-bc1d578f6d6f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.177036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.178128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.185795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfc0ad-2e36-4e4e-9655-129fa83c948d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.220871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.221752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.222151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.222337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.223553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.225536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.232592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 62/127] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.232592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.257227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c406403-1fd6-4283-916e-bc1d578f6d6f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.297192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.298167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.298559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.298840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.300081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.301677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.308326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 66/128] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.308326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.341900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94451157-dab2-4dcc-bf57-b8e568062a44 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.376818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.377723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.378014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.378300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/1f7e314d61674e539a24c3878d778b9a/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.380326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] RBAC: Authorizing `identity:create_grant(project_id=2edc2b4730164d7195bc4fc982443c5b, user_id=1f7e314d61674e539a24c3878d778b9a, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.381860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.397599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.407784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5035f-2357-49f6-a9b5-9d8941912589 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.440688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.441365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.441662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.441939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.442778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] RBAC: Adding query filter params (role.id=e5378d5e9b8948bf921c2e75bc97b128, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=f4052cebea2b4d608b8155c269d1308f) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:06.443098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.444629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.455301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 63/129] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:06 2022] PUT /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/1f7e314d61674e539a24c3878d778b9a/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:06.455301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.460046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 67/130] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/role_assignments?user.id=f4052cebea2b4d608b8155c269d1308f&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=e5378d5e9b8948bf921c2e75bc97b128 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:06.460928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.523555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.674040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af177ccb-bf16-48f5-9f60-6b32640bfe5a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.674492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d00cc-76dd-4c91-9908-9aac3700522f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.712943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.713861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.714151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.714466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/912834523da04e79b38964af43312d61/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.717534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] RBAC: Authorizing `identity:create_grant(project_id=2edc2b4730164d7195bc4fc982443c5b, user_id=912834523da04e79b38964af43312d61, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.719067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.722038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.723095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.723635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.724482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.738273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:06.747599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:06.748463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:06.749259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:06.751706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 64/131] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:06.752714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.818782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 68/132] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:06 2022] PUT /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/912834523da04e79b38964af43312d61/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:06.820156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:06.846041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.853046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:06.879484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8ac8-221f-4918-b330-2d746fec0f9a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:06.943321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e391b-f2f3-44ba-81dc-d6a52d2a447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:06.946173 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e391b-f2f3-44ba-81dc-d6a52d2a447b None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:06.947720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e391b-f2f3-44ba-81dc-d6a52d2a447b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:06.948277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e391b-f2f3-44ba-81dc-d6a52d2a447b None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/1f7e314d61674e539a24c3878d778b9a/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:06.956159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e391b-f2f3-44ba-81dc-d6a52d2a447b None admin] RBAC: Authorizing `identity:create_grant(project_id=2edc2b4730164d7195bc4fc982443c5b, user_id=1f7e314d61674e539a24c3878d778b9a, role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:06.958025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e391b-f2f3-44ba-81dc-d6a52d2a447b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:06.992969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09864e86-2a0b-4330-871f-b6fc0fe1acfb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.035806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.036822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.037295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.037768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.038833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:07.039360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.040973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.044251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 65/133] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:06 2022] PUT /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/1f7e314d61674e539a24c3878d778b9a/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:07.045819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f2fbf-f638-4c66-85db-66f010fa5409 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.046195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f2fbf-f638-4c66-85db-66f010fa5409 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.046603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f2fbf-f638-4c66-85db-66f010fa5409 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.048577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 66/134] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:07.052434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 69/135] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:07.058691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f5fd-1ee5-457d-a4b9-c5ceeb296fad None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.058691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f5fd-1ee5-457d-a4b9-c5ceeb296fad None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.058691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f5fd-1ee5-457d-a4b9-c5ceeb296fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.060267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.076486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d2b7f5fd-1ee5-457d-a4b9-c5ceeb296fad None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:07.077863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d2b7f5fd-1ee5-457d-a4b9-c5ceeb296fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:07.192684 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 67/136] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:07.209361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.242645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7f5fd-1ee5-457d-a4b9-c5ceeb296fad None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.246810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c3322-3bb3-431e-aed7-89e1ce28fbf4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.289591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.290734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.291068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.291389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.294206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.296029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.298768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 70/137] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:07 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:07.304386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.382403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.445374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7f5fd-1ee5-457d-a4b9-c5ceeb296fad None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.534463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27579572-7308-44f0-93f3-132635f62f9d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.551902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.552729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.553021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.553301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.554518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.556008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.557961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 68/138] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:07.564649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.601961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.602858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.603152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.603602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] PATH_INFO: `/v3/projects/10b68da7f8344ed7ba4f153f93683b8a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.604993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] RBAC: Authorizing `identity:get_project(project_id=10b68da7f8344ed7ba4f153f93683b8a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.606626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.608863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 71/139] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:07 2022] GET /identity/v3/projects/10b68da7f8344ed7ba4f153f93683b8a => generated 312 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:07.614652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:07.640591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.803857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fddca-2518-4759-ab53-d4767de53717 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.846896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c265b-7f74-4893-833a-d78fdb9dcd4d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:07.877495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8ba5c-56f6-49ef-8351-5307b2122d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.878185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ba5c-56f6-49ef-8351-5307b2122d59 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.878474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ba5c-56f6-49ef-8351-5307b2122d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.878759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8ba5c-56f6-49ef-8351-5307b2122d59 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.879541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8ba5c-56f6-49ef-8351-5307b2122d59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.881005 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8ba5c-56f6-49ef-8351-5307b2122d59 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:07.902913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:07.919736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f81c0-782c-4297-a816-d61335b91c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:07.919871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f81c0-782c-4297-a816-d61335b91c4b None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:07.920115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f81c0-782c-4297-a816-d61335b91c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:07.920412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f81c0-782c-4297-a816-d61335b91c4b None admin] PATH_INFO: `/v3/projects/10b68da7f8344ed7ba4f153f93683b8a/users/1f7e314d61674e539a24c3878d778b9a/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:07.922507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f81c0-782c-4297-a816-d61335b91c4b None admin] RBAC: Authorizing `identity:create_grant(project_id=10b68da7f8344ed7ba4f153f93683b8a, user_id=1f7e314d61674e539a24c3878d778b9a, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:07.924010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f81c0-782c-4297-a816-d61335b91c4b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:08.016737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 69/140] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:07 2022] POST /identity/v3/users => generated 283 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:49:08.018014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 72/141] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:07 2022] PUT /identity/v3/projects/10b68da7f8344ed7ba4f153f93683b8a/users/1f7e314d61674e539a24c3878d778b9a/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:09.054148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186d83-bbf0-490f-ac5d-fa3bcecf0960 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.054647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186d83-bbf0-490f-ac5d-fa3bcecf0960 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.054722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186d83-bbf0-490f-ac5d-fa3bcecf0960 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.056284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 70/142] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:09.064042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0da5ad-4629-4375-b72b-843ed4d1d6c5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.064940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0da5ad-4629-4375-b72b-843ed4d1d6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.065428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0da5ad-4629-4375-b72b-843ed4d1d6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.085293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2b0da5ad-4629-4375-b72b-843ed4d1d6c5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:09.086964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2b0da5ad-4629-4375-b72b-843ed4d1d6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:09.165066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 73/143] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:09.201742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80186d83-bbf0-490f-ac5d-fa3bcecf0960 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.328892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80186d83-bbf0-490f-ac5d-fa3bcecf0960 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.370350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d659cd-5baf-4ab9-9433-55b51799dc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.371757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d659cd-5baf-4ab9-9433-55b51799dc8f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.372045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d659cd-5baf-4ab9-9433-55b51799dc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.372327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d659cd-5baf-4ab9-9433-55b51799dc8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.388811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d659cd-5baf-4ab9-9433-55b51799dc8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b5d659cd-5baf-4ab9-9433-55b51799dc8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:09.396606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:09.397483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:09.398264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 71/144] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:09.405315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0da5ad-4629-4375-b72b-843ed4d1d6c5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.411526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.413003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe75e0-cc09-44e8-a69d-048684164ba6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.413306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe75e0-cc09-44e8-a69d-048684164ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.413594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe75e0-cc09-44e8-a69d-048684164ba6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.415278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 72/145] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:09.457225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf2341-ca46-40f7-b9f7-bc9c71751dd7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.457503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf2341-ca46-40f7-b9f7-bc9c71751dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.457706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf2341-ca46-40f7-b9f7-bc9c71751dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.494978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-bccf2341-ca46-40f7-b9f7-bc9c71751dd7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:09.496279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bccf2341-ca46-40f7-b9f7-bc9c71751dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:09.596525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.605837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0da5ad-4629-4375-b72b-843ed4d1d6c5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.645548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 73/146] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:09.646899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa6008-60d3-4d31-a0cd-663e67378176 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.647235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa6008-60d3-4d31-a0cd-663e67378176 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.647511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa6008-60d3-4d31-a0cd-663e67378176 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.649037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 74/147] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:09.655633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10127609-a558-4f1f-b5ee-4ea1e6868de0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.655926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10127609-a558-4f1f-b5ee-4ea1e6868de0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.656196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10127609-a558-4f1f-b5ee-4ea1e6868de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.656717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.657729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.658216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.658703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.659701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:09.660214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.661819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.680953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 74/148] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:09.686413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.692240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-10127609-a558-4f1f-b5ee-4ea1e6868de0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:09.698379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-10127609-a558-4f1f-b5ee-4ea1e6868de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:09.781884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.788210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 75/149] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:09.789501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10127609-a558-4f1f-b5ee-4ea1e6868de0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.821996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e264f-b69a-4a44-9f76-4a937386fd76 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.860153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.861049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.861335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.861617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.862850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:09.864426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:09.866308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 75/150] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:09.867006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:09.908383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10127609-a558-4f1f-b5ee-4ea1e6868de0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.931637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:09.952878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:09.954171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:09.954675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:09.955215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:09.968892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:09.978020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:09.978891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:09.980115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:09.980723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af448e-caec-4b8d-921c-795688c37959 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:09.982608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 76/151] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:09.983678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.017634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.022622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.023467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.023750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.029792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.036322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:10.044430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:10.045329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:10.046392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 76/152] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:10.047057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.109053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162a1f3-648a-4478-9065-c0432e201061 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.150716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3493c-9b7e-4ad1-8a31-0c2e824e6994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.152368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3493c-9b7e-4ad1-8a31-0c2e824e6994 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.152870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3493c-9b7e-4ad1-8a31-0c2e824e6994 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.153359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3493c-9b7e-4ad1-8a31-0c2e824e6994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.154308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978209ff-a5c6-47da-bd44-dd844a0c41aa None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.193911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.194113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.194340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.194567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.195366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:10.195676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.197137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.206766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 77/153] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.207691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.211991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.234869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3493c-9b7e-4ad1-8a31-0c2e824e6994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.237792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3493c-9b7e-4ad1-8a31-0c2e824e6994 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.272017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.319405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b34718-7acb-4031-9eaa-8ace402016c3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.355131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 77/154] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:10.356933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9996b60-d978-4c82-b9b7-403692fb9078 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.357503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9996b60-d978-4c82-b9b7-403692fb9078 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.358010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9996b60-d978-4c82-b9b7-403692fb9078 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.359620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.360200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 78/155] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:10.361124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9996b60-d978-4c82-b9b7-403692fb9078 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.366010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.366010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.368081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.368887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:10.369206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.370725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.379809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 78/156] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.380713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.452722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.479609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9996b60-d978-4c82-b9b7-403692fb9078 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.487417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0c3ee-64af-4d48-8701-77bdc9e2f5b5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.504468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.521825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.523057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.523266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b1ae6-6700-49d5-b060-24f8bc2b4fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.523785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.524116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b1ae6-6700-49d5-b060-24f8bc2b4fef None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.524403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b1ae6-6700-49d5-b060-24f8bc2b4fef None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.524666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] PATH_INFO: `/v3/users/f4052cebea2b4d608b8155c269d1308f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.524982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b1ae6-6700-49d5-b060-24f8bc2b4fef None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.525994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b1ae6-6700-49d5-b060-24f8bc2b4fef None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.526628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] RBAC: Authorizing `identity:get_user(user_id=f4052cebea2b4d608b8155c269d1308f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.528097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b1ae6-6700-49d5-b060-24f8bc2b4fef None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.528624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.530066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 79/157] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.531065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 79/158] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/users/f4052cebea2b4d608b8155c269d1308f => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.532077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.534753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf714-0502-4332-8dc0-fd93b8192015 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.535064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf714-0502-4332-8dc0-fd93b8192015 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.535553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34cf714-0502-4332-8dc0-fd93b8192015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.554366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d34cf714-0502-4332-8dc0-fd93b8192015 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:10.555720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d34cf714-0502-4332-8dc0-fd93b8192015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:10.628991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 80/159] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:10.629999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cf714-0502-4332-8dc0-fd93b8192015 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.646492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79cf4c-1902-4fe0-bf41-675e211dfa7c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.694704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.696030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.696530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.697032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] PATH_INFO: `/v3/users/912834523da04e79b38964af43312d61` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.698452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=912834523da04e79b38964af43312d61)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.700805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.703228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 80/160] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/users/912834523da04e79b38964af43312d61 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.704207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.720814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.748851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34cf714-0502-4332-8dc0-fd93b8192015 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.764326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:10.784450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.785113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.785396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.785673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.786574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=1f7e314d61674e539a24c3878d778b9a) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:10.786887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.788403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.795006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 81/161] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/role_assignments?user.id=1f7e314d61674e539a24c3878d778b9a&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.795906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.824412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264d713-0897-4e6a-acdb-a115cbebe3f0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.866214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.867525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.868077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.868588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.870068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:10.872758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:10.874825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 81/162] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:10.875852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:10.926222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b15a9c-cad5-4ced-b073-f680ff138597 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:10.979778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:10.981198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:10.981502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:10.981768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:10.983761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.006687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5603b98-aad9-4d1d-b975-c463aec22621 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.016532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:11.028427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:11.029362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:11.030152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 82/163] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:11.030819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.056848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.057872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.059456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.059955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.124642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.171541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.173826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.218030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18a4a1-7e7f-4fee-86e1-71c3f907957a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.265332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.266286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.266577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.266851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/f4052cebea2b4d608b8155c269d1308f/roles/e5378d5e9b8948bf921c2e75bc97b128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.268349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.269166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2edc2b4730164d7195bc4fc982443c5b, user_id=f4052cebea2b4d608b8155c269d1308f, role_id=e5378d5e9b8948bf921c2e75bc97b128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.270702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.291597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 82/164] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:11.292974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.336986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 83/165] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:11 2022] PUT /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b/users/f4052cebea2b4d608b8155c269d1308f/roles/e5378d5e9b8948bf921c2e75bc97b128 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:11.337909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.410478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0602c1-4c9c-44ab-8577-9a2db81972a8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.451711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.452876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.453428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.453968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.454947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:11.455489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.457080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.467798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 83/166] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.473709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.524462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.537391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.543628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4c647-9cf8-4bb9-a2a9-f45147fb76c9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.592864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ed53-5cd0-42ea-bf6a-381cdb5cb3d6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.626077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.627018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.627330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.627609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.632252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.633838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.635100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.635763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 84/167] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.636578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.637096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.637576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.639044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.640736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.641640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.642814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 84/168] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:11.658771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:11.813442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591bc13-b0f4-4c48-9c64-bbe1318c4303 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.827276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:11.849298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e729c-6fe5-4738-9761-429ee7cdb541 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:11.859561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.860604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.861140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.861727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.937294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06888643-d38f-4394-a876-53cb14e35671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:11.938305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888643-d38f-4394-a876-53cb14e35671 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.938430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888643-d38f-4394-a876-53cb14e35671 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.938608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888643-d38f-4394-a876-53cb14e35671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.939519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06888643-d38f-4394-a876-53cb14e35671 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=912834523da04e79b38964af43312d61) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:11.939814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06888643-d38f-4394-a876-53cb14e35671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:11.941436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06888643-d38f-4394-a876-53cb14e35671 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:11.953362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 85/169] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/role_assignments?user.id=912834523da04e79b38964af43312d61&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:11.966716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be1a1-4a46-4ef4-8006-53b875efac4e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.967967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be1a1-4a46-4ef4-8006-53b875efac4e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.968739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be1a1-4a46-4ef4-8006-53b875efac4e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:11.970542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 86/170] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:11.977691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2288-9ad1-47f1-bd34-d2f4c8a0ea88 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:11.978309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2288-9ad1-47f1-bd34-d2f4c8a0ea88 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:11.978795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2288-9ad1-47f1-bd34-d2f4c8a0ea88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.000101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-da4d2288-9ad1-47f1-bd34-d2f4c8a0ea88 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:12.001702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-da4d2288-9ad1-47f1-bd34-d2f4c8a0ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:12.024334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.025882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.034281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.088151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 87/171] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:12.117319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2288-9ad1-47f1-bd34-d2f4c8a0ea88 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.226082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 85/172] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:12.234722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.259610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.347389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2288-9ad1-47f1-bd34-d2f4c8a0ea88 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.404163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.405391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.405904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.406386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.418291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:12.426696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:12.427634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:12.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:12.431010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 88/173] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:12.437455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.457406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08259d-7107-4056-a146-19ab35ff1906 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.465708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.541130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.542014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.542892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.543263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.544442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.545921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.552088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 86/174] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:12.556440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fee5-0592-4e01-8644-ea74663159c3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.557293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.596439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.597538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.598190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.598694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.599970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:12.600532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.602181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.612313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 89/175] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:12.618258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.709865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.730846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad6c46-0ec7-430f-97b7-c5db0c6faafd None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.742851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:12.769848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945db18-2765-4079-a1ad-f229daea943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.771130 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945db18-2765-4079-a1ad-f229daea943f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.771536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945db18-2765-4079-a1ad-f229daea943f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.771927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945db18-2765-4079-a1ad-f229daea943f None admin] PATH_INFO: `/v3/users/1f7e314d61674e539a24c3878d778b9a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.773284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945db18-2765-4079-a1ad-f229daea943f None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e314d61674e539a24c3878d778b9a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.774858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945db18-2765-4079-a1ad-f229daea943f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.778008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 90/176] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/users/1f7e314d61674e539a24c3878d778b9a => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:12.784314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b3a3f-1491-4049-881e-2e79dcaa47da None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.784718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b3a3f-1491-4049-881e-2e79dcaa47da None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.785159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b3a3f-1491-4049-881e-2e79dcaa47da None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.786600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 91/177] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:12.787450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b3a3f-1491-4049-881e-2e79dcaa47da None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:12.792777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112224a-5d88-4ce8-9634-6c4b94ce9de3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.903971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4b4c9-74a7-4610-aa8f-d466815d4a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.904872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4b4c9-74a7-4610-aa8f-d466815d4a1e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.905292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4b4c9-74a7-4610-aa8f-d466815d4a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.905733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4b4c9-74a7-4610-aa8f-d466815d4a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.906741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b3a3f-1491-4049-881e-2e79dcaa47da None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:12.906858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4b4c9-74a7-4610-aa8f-d466815d4a1e None admin] RBAC: Adding query filter params (role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=1f7e314d61674e539a24c3878d778b9a) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:12.907317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4b4c9-74a7-4610-aa8f-d466815d4a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:12.908931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4b4c9-74a7-4610-aa8f-d466815d4a1e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:12.921905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 87/178] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/role_assignments?user.id=1f7e314d61674e539a24c3878d778b9a&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 608 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:12.927353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cac5d-d2c5-4761-92a2-5b7645c638ac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.927746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cac5d-d2c5-4761-92a2-5b7645c638ac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.928040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cac5d-d2c5-4761-92a2-5b7645c638ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.951163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:12.952309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:12.952805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:12.953359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:12.972411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f87cac5d-d2c5-4761-92a2-5b7645c638ac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:12.973669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f87cac5d-d2c5-4761-92a2-5b7645c638ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:12.978209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.038011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.039903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.156795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 92/179] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:13.161452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 88/180] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:13.171844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.197178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cac5d-d2c5-4761-92a2-5b7645c638ac None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.296344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.346237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.354584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2015232-b013-4e02-ba43-2567b8aab2ff None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.389155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cac5d-d2c5-4761-92a2-5b7645c638ac None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.400734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e12982-a563-4c51-8b1c-8c18b86501c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.401662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e12982-a563-4c51-8b1c-8c18b86501c0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.402450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e12982-a563-4c51-8b1c-8c18b86501c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.402732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e12982-a563-4c51-8b1c-8c18b86501c0 None admin] PATH_INFO: `/v3/projects/10b68da7f8344ed7ba4f153f93683b8a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.404804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e12982-a563-4c51-8b1c-8c18b86501c0 None admin] RBAC: Authorizing `identity:get_project(project_id=10b68da7f8344ed7ba4f153f93683b8a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.406277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e12982-a563-4c51-8b1c-8c18b86501c0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.408269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 93/181] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/projects/10b68da7f8344ed7ba4f153f93683b8a => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:13.409225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0c0e-b66c-4d79-af86-4938b44b51cc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.409516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0c0e-b66c-4d79-af86-4938b44b51cc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.409794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0c0e-b66c-4d79-af86-4938b44b51cc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.413421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 94/182] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:13.416476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb0c0e-b66c-4d79-af86-4938b44b51cc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.437088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36160c0-646b-474b-af43-ce2ffd1f5007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.438256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36160c0-646b-474b-af43-ce2ffd1f5007 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.438562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36160c0-646b-474b-af43-ce2ffd1f5007 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.438844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36160c0-646b-474b-af43-ce2ffd1f5007 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.452208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36160c0-646b-474b-af43-ce2ffd1f5007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c36160c0-646b-474b-af43-ce2ffd1f5007 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:13.461808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:13.462820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:13.463668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 89/183] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:13.464990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02447-f055-4a06-9489-da187ec98d55 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.464990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02447-f055-4a06-9489-da187ec98d55 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.464990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02447-f055-4a06-9489-da187ec98d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.490434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b7d02447-f055-4a06-9489-da187ec98d55 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:13.492431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b7d02447-f055-4a06-9489-da187ec98d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:13.566776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb0c0e-b66c-4d79-af86-4938b44b51cc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.579709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 90/184] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:13.580902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02447-f055-4a06-9489-da187ec98d55 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.591379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.615493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.616580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.617655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.618152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.619228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=10b68da7f8344ed7ba4f153f93683b8a, user.id=1f7e314d61674e539a24c3878d778b9a) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:13.619909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.621953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.629007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 95/185] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/role_assignments?user.id=1f7e314d61674e539a24c3878d778b9a&scope.project.id=10b68da7f8344ed7ba4f153f93683b8a&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:13.630071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.655914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.711478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02447-f055-4a06-9489-da187ec98d55 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.760260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c74dee-44af-462e-b89c-e0b088d2f87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.761142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c74dee-44af-462e-b89c-e0b088d2f87b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.761446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c74dee-44af-462e-b89c-e0b088d2f87b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.761729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c74dee-44af-462e-b89c-e0b088d2f87b None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.762514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c74dee-44af-462e-b89c-e0b088d2f87b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:13.762830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c74dee-44af-462e-b89c-e0b088d2f87b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.765061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c74dee-44af-462e-b89c-e0b088d2f87b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.778145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 91/186] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:13.778787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b168c-137d-43d4-aa30-e157ad345e2d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.783617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4bb9-20d6-4ac3-93d8-f82ffe41d060 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.784307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4bb9-20d6-4ac3-93d8-f82ffe41d060 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.784601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4bb9-20d6-4ac3-93d8-f82ffe41d060 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.790722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 92/187] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:13.791970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4bb9-20d6-4ac3-93d8-f82ffe41d060 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.835718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af245d4d-7c1d-47c9-bb5c-71f22abc29af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.836538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af245d4d-7c1d-47c9-bb5c-71f22abc29af None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.837510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af245d4d-7c1d-47c9-bb5c-71f22abc29af None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.837510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af245d4d-7c1d-47c9-bb5c-71f22abc29af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.838281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af245d4d-7c1d-47c9-bb5c-71f22abc29af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:13.840611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af245d4d-7c1d-47c9-bb5c-71f22abc29af None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:13.843508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 96/188] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:13.844616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc453-673f-4952-a1c5-7132c7c2b27e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.844847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc453-673f-4952-a1c5-7132c7c2b27e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.845131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc453-673f-4952-a1c5-7132c7c2b27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.864589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-20ddc453-673f-4952-a1c5-7132c7c2b27e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:13.866107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-20ddc453-673f-4952-a1c5-7132c7c2b27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:13.925710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.939086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4bb9-20d6-4ac3-93d8-f82ffe41d060 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:13.952533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 97/189] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:13.952533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddc453-673f-4952-a1c5-7132c7c2b27e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:13.973330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:13.984885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:13.985765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:13.986047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:13.986328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] PATH_INFO: `/v3/users/e88be696e57f4341974d99d7e96adf32` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:13.999884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] RBAC: Authorizing `identity:get_user(user_id=e88be696e57f4341974d99d7e96adf32)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.001348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.007215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 93/190] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:13 2022] GET /identity/v3/users/e88be696e57f4341974d99d7e96adf32 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:14.007215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.063932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddc453-673f-4952-a1c5-7132c7c2b27e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.101878 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.102561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.102851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.103134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.103884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.105559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.119444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd5b38-93d3-4564-8cd7-628e3a60d89a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.154727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.155567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.155856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.156133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.166182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:14.174387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:14.175493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 94/191] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:14.181409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.218297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 98/192] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:13 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:14.221342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.228635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.282873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.290116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550e1cc-d135-4b8e-9653-3aedc0f03589 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.331820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56027ded-da42-4c08-a02a-4fbd34dc4743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.332651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56027ded-da42-4c08-a02a-4fbd34dc4743 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.333016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56027ded-da42-4c08-a02a-4fbd34dc4743 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.333375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56027ded-da42-4c08-a02a-4fbd34dc4743 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.376615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252ea2b-a146-4fe6-81de-379ba6d9d534 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.428439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56027ded-da42-4c08-a02a-4fbd34dc4743 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.429490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df811f-7cec-4ff2-a962-3cb66db5bebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.430178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df811f-7cec-4ff2-a962-3cb66db5bebd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.430470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df811f-7cec-4ff2-a962-3cb66db5bebd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.430752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df811f-7cec-4ff2-a962-3cb66db5bebd None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.431412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56027ded-da42-4c08-a02a-4fbd34dc4743 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.431567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df811f-7cec-4ff2-a962-3cb66db5bebd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:14.431887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df811f-7cec-4ff2-a962-3cb66db5bebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.433829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df811f-7cec-4ff2-a962-3cb66db5bebd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.444839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 99/193] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:14.446238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae1d0b-d4e6-4740-8c44-d643ed694932 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.446531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae1d0b-d4e6-4740-8c44-d643ed694932 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.446813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae1d0b-d4e6-4740-8c44-d643ed694932 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.448081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 100/194] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:14.450267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae1d0b-d4e6-4740-8c44-d643ed694932 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.490553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.563138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 95/195] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:14.564540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076b37e-c4b7-4b81-8e23-5ffd16b9ecd0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.564899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076b37e-c4b7-4b81-8e23-5ffd16b9ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.565174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076b37e-c4b7-4b81-8e23-5ffd16b9ecd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.587121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3076b37e-c4b7-4b81-8e23-5ffd16b9ecd0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:14.588356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3076b37e-c4b7-4b81-8e23-5ffd16b9ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:14.594654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.595780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae1d0b-d4e6-4740-8c44-d643ed694932 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.640915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.641928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.642218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.642499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] PATH_INFO: `/v3/users/e88be696e57f4341974d99d7e96adf32` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.643866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] RBAC: Authorizing `identity:get_user(user_id=e88be696e57f4341974d99d7e96adf32)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.645331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.647422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 101/196] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/users/e88be696e57f4341974d99d7e96adf32 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:14.648020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.675355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 96/197] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:14.676500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076b37e-c4b7-4b81-8e23-5ffd16b9ecd0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.769295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dc4a8-2fb1-49d0-92c4-161fff56ff3d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.779034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.810513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.811624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.811913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.812364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.813599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.815082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.816822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 102/198] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:14.817318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076b37e-c4b7-4b81-8e23-5ffd16b9ecd0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.817587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:14.859811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.860476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.860818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.861107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:14.865391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:14.952788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e6c6e-96a8-4130-8203-b17fbebb3e74 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:14.955775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:14.957428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:14.990222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:14.991031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:14.991350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:14.991631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.003482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:15.012069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:15.013176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:15.014041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 103/199] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:15.014718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.037393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:15.069803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 97/200] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:15.070958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.127704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:15.136466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c44d2b-3f08-4240-ad72-67d57e325380 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.175547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.176342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.176699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.177076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.178016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=3619df4b249349909961aab754e48694, user.id=e88be696e57f4341974d99d7e96adf32) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:15.178446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.179741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646fd4f-3f13-426e-98c3-7c1bd6a4454a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.180079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.192496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 104/201] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/role_assignments?user.id=e88be696e57f4341974d99d7e96adf32&scope.project.id=3619df4b249349909961aab754e48694&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:15.192496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.259955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.260653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.261002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.261294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.265180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:15.265509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.266948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.290242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 98/202] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:15.296090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.374351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:15.394504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdb7ac-f286-4823-8655-a7139d513f3c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.437782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.438708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.439000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.439323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.440542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.442011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.450272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 105/203] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:15.450272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.480540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:15.488467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9d887-604c-4a1b-b46e-2e1bb724ad38 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.557087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.558051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.558349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.558632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] PATH_INFO: `/v3/users/f4052cebea2b4d608b8155c269d1308f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.560190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] RBAC: Authorizing `identity:get_user(user_id=f4052cebea2b4d608b8155c269d1308f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.561417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.563545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 99/204] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/users/f4052cebea2b4d608b8155c269d1308f => generated 289 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:15.569002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:15.571471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1c813-53d8-4445-9a62-c2040ceabcad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.612796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb234b99-d101-433e-b680-406acce8607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.613966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb234b99-d101-433e-b680-406acce8607b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.614554 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb234b99-d101-433e-b680-406acce8607b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.615087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb234b99-d101-433e-b680-406acce8607b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.616798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb234b99-d101-433e-b680-406acce8607b None admin] RBAC: Adding query filter params (role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=3619df4b249349909961aab754e48694, user.id=e88be696e57f4341974d99d7e96adf32) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:15.617685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb234b99-d101-433e-b680-406acce8607b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.620011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb234b99-d101-433e-b680-406acce8607b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.627756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 106/205] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/role_assignments?user.id=e88be696e57f4341974d99d7e96adf32&scope.project.id=3619df4b249349909961aab754e48694&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:15.711086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddd1da-e0fd-4b7d-a8cc-bbb9c0846661 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:15.712708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:15.748446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:15.749113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:15.749404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:15.749688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:15.823040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:15.824639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:15.954420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 100/206] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:15.960114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb234b99-d101-433e-b680-406acce8607b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.086003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.096667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb234b99-d101-433e-b680-406acce8607b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.134718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a459d-3605-4bc2-9847-bafc302da41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.135628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a459d-3605-4bc2-9847-bafc302da41e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.136543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a459d-3605-4bc2-9847-bafc302da41e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.136874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a459d-3605-4bc2-9847-bafc302da41e None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.138175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a459d-3605-4bc2-9847-bafc302da41e None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:16.139925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a459d-3605-4bc2-9847-bafc302da41e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:16.141967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 107/207] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:15 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:16.147694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.155200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.298948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70276cb4-7bf6-4b1c-855c-453ffea90f9b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.347873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953acadf-19e5-401b-9182-2ea964d4c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.348887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953acadf-19e5-401b-9182-2ea964d4c403 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.349370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953acadf-19e5-401b-9182-2ea964d4c403 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.350087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953acadf-19e5-401b-9182-2ea964d4c403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.351137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953acadf-19e5-401b-9182-2ea964d4c403 None admin] RBAC: Adding query filter params (role.id=e5378d5e9b8948bf921c2e75bc97b128, scope.project.id=2edc2b4730164d7195bc4fc982443c5b, user.id=f4052cebea2b4d608b8155c269d1308f) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:16.351672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953acadf-19e5-401b-9182-2ea964d4c403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:16.353302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953acadf-19e5-401b-9182-2ea964d4c403 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:16.363879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 101/208] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:16 2022] GET /identity/v3/role_assignments?user.id=f4052cebea2b4d608b8155c269d1308f&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=e5378d5e9b8948bf921c2e75bc97b128 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:16.511325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b624b-ff4f-4692-815f-e57822290a98 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.511877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b624b-ff4f-4692-815f-e57822290a98 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.511956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b624b-ff4f-4692-815f-e57822290a98 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.513463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 108/209] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:16.519041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04726ec8-d741-4eb6-aec5-3db554f6fb49 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.519894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04726ec8-d741-4eb6-aec5-3db554f6fb49 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.520378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04726ec8-d741-4eb6-aec5-3db554f6fb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.548371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-04726ec8-d741-4eb6-aec5-3db554f6fb49 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:16.549965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-04726ec8-d741-4eb6-aec5-3db554f6fb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:16.626596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.634735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 102/210] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:16.659793 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b624b-ff4f-4692-815f-e57822290a98 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.777327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b624b-ff4f-4692-815f-e57822290a98 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:16.813061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c6a37-5be0-4fd2-bb16-e5d4408e636f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:16.813869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6a37-5be0-4fd2-bb16-e5d4408e636f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.814154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6a37-5be0-4fd2-bb16-e5d4408e636f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.814432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c6a37-5be0-4fd2-bb16-e5d4408e636f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.815649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c6a37-5be0-4fd2-bb16-e5d4408e636f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:16.817175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c6a37-5be0-4fd2-bb16-e5d4408e636f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:16.818940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 109/211] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:16.824931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04726ec8-d741-4eb6-aec5-3db554f6fb49 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:16.836592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb09f-8d96-4f61-8a4c-b6232e4d91f5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.836899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb09f-8d96-4f61-8a4c-b6232e4d91f5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.837152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb09f-8d96-4f61-8a4c-b6232e4d91f5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.838649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 110/212] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:16.846670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4380-6222-4f6f-a98d-e2c41d797bb9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:16.846959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4380-6222-4f6f-a98d-e2c41d797bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:16.847415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b4380-6222-4f6f-a98d-e2c41d797bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:16.869148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b83b4380-6222-4f6f-a98d-e2c41d797bb9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:16.870971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b83b4380-6222-4f6f-a98d-e2c41d797bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:16.936553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:16.999223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 111/213] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:17.004234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04726ec8-d741-4eb6-aec5-3db554f6fb49 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.021965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b4380-6222-4f6f-a98d-e2c41d797bb9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.045410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.046406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.046948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.047483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.048939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:17.051123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:17.137600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b4380-6222-4f6f-a98d-e2c41d797bb9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.166034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 103/214] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:16 2022] POST /identity/v3/users => generated 297 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:17.186022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.187281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.187773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.188327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.202987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:17.211784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:17.212737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:17.213563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:17.217168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 112/215] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:17.223840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.285286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:17.338471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209fc5f-0200-4f54-bf17-02dcda825860 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.377184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.377871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.378156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.378429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.379235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:17.379515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:17.380889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:17.390098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 104/216] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:17.395377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.420161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:17.509583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322def5-f06a-4dfd-9071-ec0e8d1a921d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.548778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2eb1c-6029-40b2-b437-e00670c035f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.549662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2eb1c-6029-40b2-b437-e00670c035f2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.549943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2eb1c-6029-40b2-b437-e00670c035f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.550215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2eb1c-6029-40b2-b437-e00670c035f2 None admin] PATH_INFO: `/v3/users/4f420c03ba5a4c60a3d4131489fc7f3c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.565926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2eb1c-6029-40b2-b437-e00670c035f2 None admin] RBAC: Authorizing `identity:get_user(user_id=4f420c03ba5a4c60a3d4131489fc7f3c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:17.567479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2eb1c-6029-40b2-b437-e00670c035f2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:17.569504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 113/217] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/users/4f420c03ba5a4c60a3d4131489fc7f3c => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:17.575554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.693416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68b0e6-eb3e-4295-896c-424256523541 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:17.709122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:17.732276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:17.733383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.733910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.734485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.797265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb59188-2f5b-4306-86a8-ed3dfa071886 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.797608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb59188-2f5b-4306-86a8-ed3dfa071886 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.797930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb59188-2f5b-4306-86a8-ed3dfa071886 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.799374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 114/218] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:17.805462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e89d2-0f09-482d-b6ce-4bc39899ced1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:17.806120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e89d2-0f09-482d-b6ce-4bc39899ced1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:17.806766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e89d2-0f09-482d-b6ce-4bc39899ced1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:17.827703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b29e89d2-0f09-482d-b6ce-4bc39899ced1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:17.830114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b29e89d2-0f09-482d-b6ce-4bc39899ced1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:17.830724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:17.832254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:17.906998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 115/219] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:17.932312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e89d2-0f09-482d-b6ce-4bc39899ced1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:17.935916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.050576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 105/220] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:18.056630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.078074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.090241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e89d2-0f09-482d-b6ce-4bc39899ced1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.140841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e446d28-6fbc-4010-9a2f-2fea6028f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.141643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e446d28-6fbc-4010-9a2f-2fea6028f316 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.141931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e446d28-6fbc-4010-9a2f-2fea6028f316 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.142740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e446d28-6fbc-4010-9a2f-2fea6028f316 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.159255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e446d28-6fbc-4010-9a2f-2fea6028f316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-9e446d28-6fbc-4010-9a2f-2fea6028f316 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:18.169319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:18.170266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:18.171063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:18.171716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 116/221] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:18.175216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaeed0d-2b3b-4d73-8dab-c68d3f769c29 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.175216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaeed0d-2b3b-4d73-8dab-c68d3f769c29 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.175216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaeed0d-2b3b-4d73-8dab-c68d3f769c29 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.175216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 117/222] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:18.178069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaeed0d-2b3b-4d73-8dab-c68d3f769c29 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.202636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a84bc3-32a8-4848-854e-6bf2dc7a126e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.246986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.249380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e9478-d918-4c1e-aa78-1719d4f9f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.250323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e9478-d918-4c1e-aa78-1719d4f9f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.250671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e9478-d918-4c1e-aa78-1719d4f9f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.251023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e9478-d918-4c1e-aa78-1719d4f9f8e3 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.252621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e9478-d918-4c1e-aa78-1719d4f9f8e3 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:18.254648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e9478-d918-4c1e-aa78-1719d4f9f8e3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:18.261245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 106/223] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:18 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:18.262698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d175-3ca7-42a5-93f6-0366c48ae56e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.263055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d175-3ca7-42a5-93f6-0366c48ae56e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.263903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767d175-3ca7-42a5-93f6-0366c48ae56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.287559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e767d175-3ca7-42a5-93f6-0366c48ae56e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:18.289016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e767d175-3ca7-42a5-93f6-0366c48ae56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:18.293341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaeed0d-2b3b-4d73-8dab-c68d3f769c29 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.329224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.330319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.330855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.331351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.332301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:18.332802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:18.334324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:18.344345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 118/224] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:18.350002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.379214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 107/225] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:18.379214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d175-3ca7-42a5-93f6-0366c48ae56e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.389815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.492827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974c918-e7d0-479c-97b5-af3a24acb45c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.496626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767d175-3ca7-42a5-93f6-0366c48ae56e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.532198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.533306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.534434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.534860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.535302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] PATH_INFO: `/v3/users/e88be696e57f4341974d99d7e96adf32` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.536675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] RBAC: Authorizing `identity:get_user(user_id=e88be696e57f4341974d99d7e96adf32)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:18.538518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:18.541237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 108/226] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:18 2022] GET /identity/v3/users/e88be696e57f4341974d99d7e96adf32 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:18.546275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.546728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.547442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.547745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.548100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.548946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=3619df4b249349909961aab754e48694, user.id=4f420c03ba5a4c60a3d4131489fc7f3c) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:18.549470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:18.550988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:18.559121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 119/227] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:18 2022] GET /identity/v3/role_assignments?user.id=4f420c03ba5a4c60a3d4131489fc7f3c&scope.project.id=3619df4b249349909961aab754e48694&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:18.564031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.706622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.733407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c6a34-3dd3-4765-bc94-958e2d628eeb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.758275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563822cf-117a-45f5-91a8-bd7d694f886d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:18.773958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.775266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.775753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.776223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.777887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:18.779604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:18.781830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 120/228] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:18 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:18.792249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.816134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:18.816971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:18.817262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:18.817605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:18.831787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:18.840884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:18.841805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:18.842702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 109/229] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:18.851748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:18.873587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:18.983655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a773a3-51e5-4ed7-82fc-45e93304bad5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:19.014452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff60da7-2d8b-4e41-88e5-9913742020f1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:19.027946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:19.028866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.029153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.029427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694/users/e88be696e57f4341974d99d7e96adf32/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.031451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] RBAC: Authorizing `identity:create_grant(project_id=3619df4b249349909961aab754e48694, user_id=e88be696e57f4341974d99d7e96adf32, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:19.032924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:19.051675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:19.052466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.052761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.053038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.064271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:19.064271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:19.064271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:19.064884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 110/230] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:19.070399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:19.073333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:19.127515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 121/231] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:18 2022] PUT /identity/v3/projects/3619df4b249349909961aab754e48694/users/e88be696e57f4341974d99d7e96adf32/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:19.195457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762e24f-4d1d-4b41-af70-f2cd9e8098be None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:19.240558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:19.241379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.241665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.241944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] PATH_INFO: `/v3/users/e88be696e57f4341974d99d7e96adf32` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.243288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] RBAC: Authorizing `identity:get_user(user_id=e88be696e57f4341974d99d7e96adf32)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:19.245073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:19.247525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 111/232] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:19 2022] GET /identity/v3/users/e88be696e57f4341974d99d7e96adf32 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:19.253157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:19.413499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e9fcf-b970-4fa8-b568-20b28b4ed54b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:19.435868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:19.455444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c06ba-550f-4852-8700-703c0c0a3583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:19.456706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c06ba-550f-4852-8700-703c0c0a3583 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.457200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c06ba-550f-4852-8700-703c0c0a3583 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.457704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c06ba-550f-4852-8700-703c0c0a3583 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.459141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c06ba-550f-4852-8700-703c0c0a3583 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:19.460892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c06ba-550f-4852-8700-703c0c0a3583 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:19.463224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 122/233] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:19 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:19.469058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:19.562433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:19.614837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d890e-f981-46e0-b724-8eb49f1f3f64 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:19.660013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597bcd0-a7a5-440e-bdc9-37c322baa923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:19.660929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597bcd0-a7a5-440e-bdc9-37c322baa923 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.661214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597bcd0-a7a5-440e-bdc9-37c322baa923 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.661490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597bcd0-a7a5-440e-bdc9-37c322baa923 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694/users/e88be696e57f4341974d99d7e96adf32/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.663732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597bcd0-a7a5-440e-bdc9-37c322baa923 None admin] RBAC: Authorizing `identity:create_grant(project_id=3619df4b249349909961aab754e48694, user_id=e88be696e57f4341974d99d7e96adf32, role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:19.665174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597bcd0-a7a5-440e-bdc9-37c322baa923 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:19.747814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 112/234] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:19 2022] PUT /identity/v3/projects/3619df4b249349909961aab754e48694/users/e88be696e57f4341974d99d7e96adf32/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:19.859641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb13fd4-187b-478b-9586-8fb43aca462f None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.865416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb13fd4-187b-478b-9586-8fb43aca462f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.865416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb13fd4-187b-478b-9586-8fb43aca462f None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.865416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 123/235] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:19.868361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059de317-5965-4543-a6c7-babf56d36619 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.868666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059de317-5965-4543-a6c7-babf56d36619 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.868951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059de317-5965-4543-a6c7-babf56d36619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.895632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-059de317-5965-4543-a6c7-babf56d36619 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:19.897169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-059de317-5965-4543-a6c7-babf56d36619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:19.934582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991604-e5a8-4f33-bb1e-db3b52c8f6b4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.934928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991604-e5a8-4f33-bb1e-db3b52c8f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.935155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991604-e5a8-4f33-bb1e-db3b52c8f6b4 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.936653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 124/236] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:19.942587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa6399-df86-41aa-817d-bfa7ef501323 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:19.943085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa6399-df86-41aa-817d-bfa7ef501323 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:19.943200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa6399-df86-41aa-817d-bfa7ef501323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:19.975653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-25aa6399-df86-41aa-817d-bfa7ef501323 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:19.976858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-25aa6399-df86-41aa-817d-bfa7ef501323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:19.988525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 113/237] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:20.017964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059de317-5965-4543-a6c7-babf56d36619 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.052121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 125/238] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:20.081239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa6399-df86-41aa-817d-bfa7ef501323 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.084006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.134130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059de317-5965-4543-a6c7-babf56d36619 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.175212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.176158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.176446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.176880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.178059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.179540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.186688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 114/239] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:20.187613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.294879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee453e0b-6c33-4c22-a72d-e04f05833e69 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.327515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.332658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.333345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.333651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.334353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.335254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.336718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.375095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa6399-df86-41aa-817d-bfa7ef501323 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.393665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:20.418262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 115/240] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:20 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:20.426073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.427266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.427768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.428228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.450545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:20.467581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:20.468610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:20.469444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:20.471621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 126/241] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:20.478173 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.624162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc36811-7f27-4825-b8cc-532c1ec2074d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.660964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.661570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.661857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.662139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.662902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:20.663246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.664667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.674732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 116/242] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:20.679979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.792423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66c4e-4ed8-4d97-9141-fa9947e81cda None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:20.841440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3b1d-9848-41a2-b2d1-ffcd4b61be02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:20.842335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3b1d-9848-41a2-b2d1-ffcd4b61be02 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:20.842623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3b1d-9848-41a2-b2d1-ffcd4b61be02 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:20.842905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3b1d-9848-41a2-b2d1-ffcd4b61be02 None admin] PATH_INFO: `/v3/users/b9f02a000c504ebd8c07bf8da50ed995` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:20.861837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c3b1d-9848-41a2-b2d1-ffcd4b61be02 None admin] RBAC: Authorizing `identity:get_user(user_id=b9f02a000c504ebd8c07bf8da50ed995)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:20.863400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c3b1d-9848-41a2-b2d1-ffcd4b61be02 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:20.867595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 127/243] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/users/b9f02a000c504ebd8c07bf8da50ed995 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:20.870841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:20.937515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.002036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdf14d-359c-4e86-af3e-9ad6c469a705 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.039494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.040164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.040448 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.040790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.114927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.116808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.166746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.172835 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c07dd-0bdd-475c-abb0-32bc93127830 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.173141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c07dd-0bdd-475c-abb0-32bc93127830 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.173410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c07dd-0bdd-475c-abb0-32bc93127830 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.174969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 128/244] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:21.180514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f244-66f9-4708-9062-b4263b48b936 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.180804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f244-66f9-4708-9062-b4263b48b936 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.183861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4f244-66f9-4708-9062-b4263b48b936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.219632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9ac4f244-66f9-4708-9062-b4263b48b936 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:21.221266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9ac4f244-66f9-4708-9062-b4263b48b936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:21.229502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 117/245] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:21.233626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.258495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.308854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 129/246] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:21.332507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4f244-66f9-4708-9062-b4263b48b936 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.343442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b9232-3a98-40b3-be6a-5fc8fbb33fe6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.382172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.383156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.383478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.383767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.385155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.386603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.388678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 118/247] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:21.396520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.420925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.467743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4f244-66f9-4708-9062-b4263b48b936 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.513030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.514328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.515013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.517088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.530180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:21.538750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:21.539707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:21.540527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:21.544479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 130/248] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:21.552211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.554208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7abf6-cc7f-41ca-bef7-df383fdd0e78 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.593685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.619926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc62de-8d5b-4fe2-9274-853986f04ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.621166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc62de-8d5b-4fe2-9274-853986f04ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.621932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc62de-8d5b-4fe2-9274-853986f04ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.622370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc62de-8d5b-4fe2-9274-853986f04ef1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.623392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc62de-8d5b-4fe2-9274-853986f04ef1 None admin] RBAC: Adding query filter params (role.id=e5378d5e9b8948bf921c2e75bc97b128, scope.project.id=3619df4b249349909961aab754e48694, user.id=b9f02a000c504ebd8c07bf8da50ed995) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:21.623859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc62de-8d5b-4fe2-9274-853986f04ef1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.625485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc62de-8d5b-4fe2-9274-853986f04ef1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.636478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 119/249] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/role_assignments?user.id=b9f02a000c504ebd8c07bf8da50ed995&scope.project.id=3619df4b249349909961aab754e48694&role.id=e5378d5e9b8948bf921c2e75bc97b128 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:21.638077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b048aad-2848-436d-ab95-ae563a5a1b74 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.638573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b048aad-2848-436d-ab95-ae563a5a1b74 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.639059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b048aad-2848-436d-ab95-ae563a5a1b74 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.640824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 120/250] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:21.653523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860da9fc-29c7-4267-b361-62496b870291 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.654181 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860da9fc-29c7-4267-b361-62496b870291 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.654672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860da9fc-29c7-4267-b361-62496b870291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.688509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-860da9fc-29c7-4267-b361-62496b870291 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:21.690095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-860da9fc-29c7-4267-b361-62496b870291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:21.786525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 121/251] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:21.821275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ee801-eef6-4c72-8ab3-adaf67c2920d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:21.860292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860da9fc-29c7-4267-b361-62496b870291 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:21.880590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:21.897351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:21.898033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:21.898322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:21.898600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:21.899468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:21.902784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:21.902784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:21.920038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 131/252] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:21.925654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.045424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860da9fc-29c7-4267-b361-62496b870291 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.105523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.106343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.106630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.106909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.124232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.126022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:22.140483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:22.141456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:22.142310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:22.143489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 122/253] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:22.147788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.164197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d292-2009-498a-a8ad-74f7b9ffe408 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.246884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.252618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.252911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.253195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] PATH_INFO: `/v3/users/4f420c03ba5a4c60a3d4131489fc7f3c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.257141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] RBAC: Authorizing `identity:get_user(user_id=4f420c03ba5a4c60a3d4131489fc7f3c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.258616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.260303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 132/254] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:21 2022] GET /identity/v3/users/4f420c03ba5a4c60a3d4131489fc7f3c => generated 297 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:22.275916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.284223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.361927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45774086-43a8-4345-bb22-fe51c9b3dd9d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.412215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c83e6-8af7-465d-a6fb-249bde14db55 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.454717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.455621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.455909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.456192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.459905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.461375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.463099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 133/255] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:22.464628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.465371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.465654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.465933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.466748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:22.467060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.468518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.468940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.484992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 123/256] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:22.496780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.513302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.603050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124c8e-9d4d-4d9e-9cea-c49f30114b3d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.642258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5951e-b9b3-412e-8b57-4d984c9b4749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.643082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5951e-b9b3-412e-8b57-4d984c9b4749 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.643394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5951e-b9b3-412e-8b57-4d984c9b4749 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.643683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5951e-b9b3-412e-8b57-4d984c9b4749 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694/users/4f420c03ba5a4c60a3d4131489fc7f3c/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.646533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5951e-b9b3-412e-8b57-4d984c9b4749 None admin] RBAC: Authorizing `identity:create_grant(project_id=3619df4b249349909961aab754e48694, user_id=4f420c03ba5a4c60a3d4131489fc7f3c, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.648022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5951e-b9b3-412e-8b57-4d984c9b4749 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.711331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bc56c-82ff-42ee-8c9a-03297a910d9b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:22.731090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 134/257] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:22 2022] PUT /identity/v3/projects/3619df4b249349909961aab754e48694/users/4f420c03ba5a4c60a3d4131489fc7f3c/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:22.757856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea91af0-2274-4905-82fc-3823d496cd56 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.758205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea91af0-2274-4905-82fc-3823d496cd56 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.758429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea91af0-2274-4905-82fc-3823d496cd56 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.760428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 135/258] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:22.768328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:22.770247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f492-6df0-45c0-9350-436406c7757b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.770799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f492-6df0-45c0-9350-436406c7757b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.771094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f492-6df0-45c0-9350-436406c7757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.786387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:22.786801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:22.787069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:22.787376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] PATH_INFO: `/v3/users/e88be696e57f4341974d99d7e96adf32` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:22.790240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] RBAC: Authorizing `identity:get_user(user_id=e88be696e57f4341974d99d7e96adf32)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:22.792417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:22.795102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 124/259] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/users/e88be696e57f4341974d99d7e96adf32 => generated 283 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:22.800801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:22.807594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6ef6f492-6df0-45c0-9350-436406c7757b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:22.808969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6ef6f492-6df0-45c0-9350-436406c7757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:22.994679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 136/260] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:23.019927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6f492-6df0-45c0-9350-436406c7757b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.035061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab752-2257-46cd-9bd7-761dc2855e72 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.063565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.100205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.100882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.101171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.101460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.141565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6f492-6df0-45c0-9350-436406c7757b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.190347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.191089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.191397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.191675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.214857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:23.236161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:23.237104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:23.237900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 137/261] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:23.246912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.251311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.254180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.255831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.431566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.471447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05640f9e-6455-40be-b5ed-72c080a168fb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.500156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 125/262] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:23.505921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.512160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.512160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.512567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.512731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.513511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:23.513826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.515588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.530180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 138/263] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:23.535821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.650792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f783c07-76e6-487d-8f5a-8598b5c6df3d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.688209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.689097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.689673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.689959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] PATH_INFO: `/v3/users/e88be696e57f4341974d99d7e96adf32` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.691207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] RBAC: Authorizing `identity:get_user(user_id=e88be696e57f4341974d99d7e96adf32)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.692808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.694588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 139/264] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/users/e88be696e57f4341974d99d7e96adf32 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:23.703532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.707847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f0156-3ea7-41da-96fd-b5f074a71c9f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.708853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.711708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:23.790102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.791370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.792449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.793077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:23.794511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:23.796307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:23.803650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 126/265] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:23.809220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:23.823764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bfca9-e5fd-4482-be31-16351584cd39 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:23.895083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:23.895787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:23.896077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:23.896361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.013851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a76534-bc01-4e8c-a857-c65ce4c97bde None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.037153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:24.038626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:24.041526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:24.080145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8a582-c5ec-4117-baf5-a4ca694e1f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.080825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8a582-c5ec-4117-baf5-a4ca694e1f67 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.081095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8a582-c5ec-4117-baf5-a4ca694e1f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.081485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8a582-c5ec-4117-baf5-a4ca694e1f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.082339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8a582-c5ec-4117-baf5-a4ca694e1f67 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=3619df4b249349909961aab754e48694, user.id=e88be696e57f4341974d99d7e96adf32) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:24.082660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8a582-c5ec-4117-baf5-a4ca694e1f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:24.084128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8a582-c5ec-4117-baf5-a4ca694e1f67 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:24.094449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 127/266] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/role_assignments?user.id=e88be696e57f4341974d99d7e96adf32&scope.project.id=3619df4b249349909961aab754e48694&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:24.095825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0732be-94f2-4c5d-a689-d8f44f552b3a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.096120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0732be-94f2-4c5d-a689-d8f44f552b3a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.096403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0732be-94f2-4c5d-a689-d8f44f552b3a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.097646 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 128/267] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:24.103536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1be55-fcb2-4b0f-9e18-4cd99ebc5eca None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.103850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1be55-fcb2-4b0f-9e18-4cd99ebc5eca None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.104110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1be55-fcb2-4b0f-9e18-4cd99ebc5eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.134685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fbf1be55-fcb2-4b0f-9e18-4cd99ebc5eca None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:24.136613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fbf1be55-fcb2-4b0f-9e18-4cd99ebc5eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:24.192195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:24.244125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 140/268] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:24.252314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.287056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 129/269] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:24.380643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1be55-fcb2-4b0f-9e18-4cd99ebc5eca None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.430623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6a230-1b89-4abd-8f8c-e716255b35d6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.433080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:24.497738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a8760-cf59-4556-8733-ec2f2952fd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.498742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a8760-cf59-4556-8733-ec2f2952fd9e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.499037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a8760-cf59-4556-8733-ec2f2952fd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.499364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a8760-cf59-4556-8733-ec2f2952fd9e None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.502278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a8760-cf59-4556-8733-ec2f2952fd9e None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:24.503916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a8760-cf59-4556-8733-ec2f2952fd9e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:24.505772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 141/270] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:24.507123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f5d97-a8a8-4760-9839-65bee81e7702 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.507444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f5d97-a8a8-4760-9839-65bee81e7702 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.507722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f5d97-a8a8-4760-9839-65bee81e7702 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.508969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 142/271] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:24.515383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f5d97-a8a8-4760-9839-65bee81e7702 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.537105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1be55-fcb2-4b0f-9e18-4cd99ebc5eca None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.584668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac566-eb2a-479b-afc7-9dabff3479d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.585629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac566-eb2a-479b-afc7-9dabff3479d9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.585925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac566-eb2a-479b-afc7-9dabff3479d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.586214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac566-eb2a-479b-afc7-9dabff3479d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.601466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac566-eb2a-479b-afc7-9dabff3479d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-041ac566-eb2a-479b-afc7-9dabff3479d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:24.614121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:24.618337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 130/272] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:24.619153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984997f-9bef-4eaf-b235-dc17599ac9e4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.619972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984997f-9bef-4eaf-b235-dc17599ac9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.619972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984997f-9bef-4eaf-b235-dc17599ac9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.640596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0984997f-9bef-4eaf-b235-dc17599ac9e4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:24.642432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0984997f-9bef-4eaf-b235-dc17599ac9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:24.663463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:24.674151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f5d97-a8a8-4760-9839-65bee81e7702 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.727602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.728271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.728806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.728989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.730454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] RBAC: Adding query filter params (role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=3619df4b249349909961aab754e48694, user.id=e88be696e57f4341974d99d7e96adf32) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:24.730690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:24.732171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:24.738696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 143/273] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/role_assignments?user.id=e88be696e57f4341974d99d7e96adf32&scope.project.id=3619df4b249349909961aab754e48694&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:24.739601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.750130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 131/274] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:24.751976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04720382-ac75-4811-8c8c-6093c012a461 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.752587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04720382-ac75-4811-8c8c-6093c012a461 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.753090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04720382-ac75-4811-8c8c-6093c012a461 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.754793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 132/275] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:24.761816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024c8da-361f-44f8-b2ab-4483b9fe4a11 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.762436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024c8da-361f-44f8-b2ab-4483b9fe4a11 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.762941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024c8da-361f-44f8-b2ab-4483b9fe4a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.785869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6024c8da-361f-44f8-b2ab-4483b9fe4a11 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:24.787967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6024c8da-361f-44f8-b2ab-4483b9fe4a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:24.807505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:24.870245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 133/276] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:24.879695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024c8da-361f-44f8-b2ab-4483b9fe4a11 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.880616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27952569-371c-43bc-8a32-e6b181b7d255 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:24.924427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:24.925111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:24.925563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:24.925849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:24.926618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:24.926929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:24.928355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:24.939699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 144/277] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:24.945772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:24.995432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.018890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024c8da-361f-44f8-b2ab-4483b9fe4a11 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.063197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.064005 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.064296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.064575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.065864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.067489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.069111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.070453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 134/278] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:25.071264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.079666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32daea-dc06-4c7b-8a81-cc208e6141b0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.117736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.118916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.119228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.120413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.130845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:25.139090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:25.140035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:25.140852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 145/279] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:25.141521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.187171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aaddd-bdc1-4b95-9e6e-a33231eea943 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.234304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.235497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.236428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.236900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] PATH_INFO: `/v3/users/4adaa81489664dc9be39ed4830804e46` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.248559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb849933-2f24-467c-9524-1ebc2aa94d8a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.254961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] RBAC: Authorizing `identity:get_user(user_id=4adaa81489664dc9be39ed4830804e46)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.256577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.258459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 135/280] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/users/4adaa81489664dc9be39ed4830804e46 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:25.264058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.275738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.286042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.286723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.287006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.287322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.288040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.289551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.367647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb7406-04db-49d1-b7aa-d1e4630f83d6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.401790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.402628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.402919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.403227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.403978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:25.404290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.405662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.413688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 136/281] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:25.415775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.422023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 146/282] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:25 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:25.422859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.441436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.493178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.531532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc8929-5592-4726-a40e-d36324914783 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.542622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db8d2d-4e47-442f-9d9a-72eca0863663 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.571636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4d46a-ea10-4d20-98d0-79ce61e2a015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.572301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d46a-ea10-4d20-98d0-79ce61e2a015 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.572590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d46a-ea10-4d20-98d0-79ce61e2a015 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.573523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d46a-ea10-4d20-98d0-79ce61e2a015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.574463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4d46a-ea10-4d20-98d0-79ce61e2a015 None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, user.id=4adaa81489664dc9be39ed4830804e46, scope.system=all) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:25.574804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4d46a-ea10-4d20-98d0-79ce61e2a015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.576275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4d46a-ea10-4d20-98d0-79ce61e2a015 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.582411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 137/283] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/role_assignments?user.id=4adaa81489664dc9be39ed4830804e46&scope.system=all&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:25.582548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.583849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a6c1-a206-4e6c-af07-8ab27210e294 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.583849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a6c1-a206-4e6c-af07-8ab27210e294 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.584118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311a6c1-a206-4e6c-af07-8ab27210e294 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.585329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 138/284] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:25.586372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.590286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.590525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] PATH_INFO: `/v3/users/b9f02a000c504ebd8c07bf8da50ed995` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.591612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533e666-dc67-4823-a96d-177432e1aa7c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.591907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533e666-dc67-4823-a96d-177432e1aa7c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.592191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533e666-dc67-4823-a96d-177432e1aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.596081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] RBAC: Authorizing `identity:get_user(user_id=b9f02a000c504ebd8c07bf8da50ed995)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.597588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.599760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 147/285] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/users/b9f02a000c504ebd8c07bf8da50ed995 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:25.606309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.614312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9533e666-dc67-4823-a96d-177432e1aa7c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:25.615631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9533e666-dc67-4823-a96d-177432e1aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:25.720809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 139/286] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:25.730894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.749547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533e666-dc67-4823-a96d-177432e1aa7c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.759707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f135-4261-422d-b4ec-4b61640b4e7a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.804889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.805830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.806159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.806438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.808611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.810110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:25.812487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 148/287] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:25.817972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.834965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:25.882525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533e666-dc67-4823-a96d-177432e1aa7c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.928089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.928913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.929207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.929493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.941935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:25.944833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e19a62-51a7-41c0-8f03-f62100e1a7f3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:25.950277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:25.952967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:25.953764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 140/288] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:25.957581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:25.992729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:25.993528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:25.993812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:25.994094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694/users/b9f02a000c504ebd8c07bf8da50ed995/roles/e5378d5e9b8948bf921c2e75bc97b128` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:25.996931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] RBAC: Authorizing `identity:create_grant(project_id=3619df4b249349909961aab754e48694, user_id=b9f02a000c504ebd8c07bf8da50ed995, role_id=e5378d5e9b8948bf921c2e75bc97b128)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:25.998385 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:26.077549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192e4f9-5b6c-411c-853b-2c603142b4dd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.088725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 149/289] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:49:25 2022] PUT /identity/v3/projects/3619df4b249349909961aab754e48694/users/b9f02a000c504ebd8c07bf8da50ed995/roles/e5378d5e9b8948bf921c2e75bc97b128 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:26.103613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.117481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.118180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.118470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.118754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.119619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:26.120101 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:26.121782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:26.132819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 141/290] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:26.138996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.161570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.285567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708839a-82a6-436c-a3d8-7c74bf41d1ad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.323639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.324527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.324809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.325085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] PATH_INFO: `/v3/users/4f420c03ba5a4c60a3d4131489fc7f3c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.326302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] RBAC: Authorizing `identity:get_user(user_id=4f420c03ba5a4c60a3d4131489fc7f3c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:26.327783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:26.329760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 150/291] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/users/4f420c03ba5a4c60a3d4131489fc7f3c => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:26.334945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.467217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dceef9-580c-42d8-a88c-dbc71ba70840 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:26.511203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:26.511203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:26.511203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:26.511203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:26.563652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.580720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:26.582226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:26.707535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 142/292] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:26.716735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:26.888392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:26.945442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6bdd7-3d36-4995-a131-a217db7616be None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.023464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a712a-9561-4d05-a3cc-671245c8ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.024357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a712a-9561-4d05-a3cc-671245c8ed01 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.024641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a712a-9561-4d05-a3cc-671245c8ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.024918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a712a-9561-4d05-a3cc-671245c8ed01 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.026225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a712a-9561-4d05-a3cc-671245c8ed01 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.028008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a712a-9561-4d05-a3cc-671245c8ed01 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.030135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 151/293] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:26 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:27.040346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:27.211621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:27.236315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ab6a4-3bff-4e23-a78c-48ca465d98d5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:27.308780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a38f-66d7-4a8d-a4d0-893a1b0e8dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:27.309453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a38f-66d7-4a8d-a4d0-893a1b0e8dba None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:27.309738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a38f-66d7-4a8d-a4d0-893a1b0e8dba None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:27.310015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a38f-66d7-4a8d-a4d0-893a1b0e8dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:27.310843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a38f-66d7-4a8d-a4d0-893a1b0e8dba None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=3619df4b249349909961aab754e48694, user.id=4f420c03ba5a4c60a3d4131489fc7f3c) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:27.311148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a38f-66d7-4a8d-a4d0-893a1b0e8dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:27.312667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a38f-66d7-4a8d-a4d0-893a1b0e8dba None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:27.324046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 143/294] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:27 2022] GET /identity/v3/role_assignments?user.id=4f420c03ba5a4c60a3d4131489fc7f3c&scope.project.id=3619df4b249349909961aab754e48694&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:28.068269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d800c4-a468-406b-babd-fdacf40a8f0e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.069406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d800c4-a468-406b-babd-fdacf40a8f0e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.069900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d800c4-a468-406b-babd-fdacf40a8f0e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.071512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 152/295] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:28.077197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb1991-4800-40b5-98ec-037b8d4e5db4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.078271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb1991-4800-40b5-98ec-037b8d4e5db4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.078760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb1991-4800-40b5-98ec-037b8d4e5db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.097572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-38cb1991-4800-40b5-98ec-037b8d4e5db4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:28.099382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-38cb1991-4800-40b5-98ec-037b8d4e5db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:28.129251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5d17d-f802-4f9c-be94-6c12a16f88cf None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.129574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5d17d-f802-4f9c-be94-6c12a16f88cf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.129808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5d17d-f802-4f9c-be94-6c12a16f88cf None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.131327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 153/296] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:28.139217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528abd-8350-4199-ac9d-dc16d886c280 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.139738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528abd-8350-4199-ac9d-dc16d886c280 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.140243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528abd-8350-4199-ac9d-dc16d886c280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.160119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-89528abd-8350-4199-ac9d-dc16d886c280 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:28.161674 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-89528abd-8350-4199-ac9d-dc16d886c280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:28.180322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 144/297] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:28.204260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb1991-4800-40b5-98ec-037b8d4e5db4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.244150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 154/298] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:28.260956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e797-64ff-4a49-b229-94ddd64e7004 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.261295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e797-64ff-4a49-b229-94ddd64e7004 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.261574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e797-64ff-4a49-b229-94ddd64e7004 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.263033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 155/299] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:28.268002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9ed09-c4e6-41bc-87fe-9281b378a0e6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.268290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9ed09-c4e6-41bc-87fe-9281b378a0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.268569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9ed09-c4e6-41bc-87fe-9281b378a0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.285095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0ac9ed09-c4e6-41bc-87fe-9281b378a0e6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:28.286262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0ac9ed09-c4e6-41bc-87fe-9281b378a0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:28.299892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.321751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb1991-4800-40b5-98ec-037b8d4e5db4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.358673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 156/300] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:28.358872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.359380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.359669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.359946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.360867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9ed09-c4e6-41bc-87fe-9281b378a0e6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.364877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.371387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:28.379336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:28.383028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 145/301] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:28.383895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.475482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9ed09-c4e6-41bc-87fe-9281b378a0e6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.489092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aff65-57f9-4490-98d6-620028090140 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.513885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.514685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.514967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.515267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.524335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.525129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.525412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.525686 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.527001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:28.527526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.528982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.530606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 146/302] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:28.531415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:28.536122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:28.537018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:28.537814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 157/303] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:28.538505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.544267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.608873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.647975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a5cbc-db73-4f42-bd61-c6d985b80b7a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.650443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66223088-ac72-4783-b521-2e73fd80692f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.685116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.685492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.686165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.686451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.686727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] PATH_INFO: `/v3/groups` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.686931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.687413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.687526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.687972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.688877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.689093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:28.689567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.691094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.700169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 147/304] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:28.708388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.767977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.769172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 158/305] 10.222.0.43 () {60 vars in 1134 bytes} [Thu Jun 23 10:49:28 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:28.770332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.818639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8a13-81c6-4c62-b25a-5d980ae5f1fa None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.863830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.865123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.865682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.865976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.866767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:28.867089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.868978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.883445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 148/306] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:28.889710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:28.915117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bd1d1-0e5a-4deb-9a08-fb3ed766359f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:28.943464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:28.956817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb8131-1b83-4972-883f-4220fef4e279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:28.957999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8131-1b83-4972-883f-4220fef4e279 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.958531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8131-1b83-4972-883f-4220fef4e279 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.959039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb8131-1b83-4972-883f-4220fef4e279 None admin] PATH_INFO: `/v3/users/4c5cf2cca21343199b7f0516d216bf48` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.975898 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb8131-1b83-4972-883f-4220fef4e279 None admin] RBAC: Authorizing `identity:get_user(user_id=4c5cf2cca21343199b7f0516d216bf48)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:28.977412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb8131-1b83-4972-883f-4220fef4e279 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:28.979917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 159/307] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/users/4c5cf2cca21343199b7f0516d216bf48 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:28.980858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994a10e-3630-477e-9b56-6c006213a29b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:28.981147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994a10e-3630-477e-9b56-6c006213a29b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:28.982477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994a10e-3630-477e-9b56-6c006213a29b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:28.982775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 160/308] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:28.986192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994a10e-3630-477e-9b56-6c006213a29b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.009199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b0123-12ef-4be4-8a51-f49fa0a47eb5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.040571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.049512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1bbb0-4202-47d2-97a6-acb37e837e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.051091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1bbb0-4202-47d2-97a6-acb37e837e7e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.051880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1bbb0-4202-47d2-97a6-acb37e837e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.056864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1bbb0-4202-47d2-97a6-acb37e837e7e None admin] PATH_INFO: `/v3/users/4adaa81489664dc9be39ed4830804e46` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.056864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1bbb0-4202-47d2-97a6-acb37e837e7e None admin] RBAC: Authorizing `identity:get_user(user_id=4adaa81489664dc9be39ed4830804e46)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.056864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1bbb0-4202-47d2-97a6-acb37e837e7e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.057980 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 149/309] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/users/4adaa81489664dc9be39ed4830804e46 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:29.060234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d347c-fcd9-4a5c-a7dc-aefcc9109561 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.060530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d347c-fcd9-4a5c-a7dc-aefcc9109561 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.060861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d347c-fcd9-4a5c-a7dc-aefcc9109561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.079281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-707d347c-fcd9-4a5c-a7dc-aefcc9109561 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:29.080532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-707d347c-fcd9-4a5c-a7dc-aefcc9109561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:29.105823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994a10e-3630-477e-9b56-6c006213a29b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.143082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.143826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.144112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.144391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.145263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] RBAC: Adding query filter params (role.id=e5378d5e9b8948bf921c2e75bc97b128, user.id=4c5cf2cca21343199b7f0516d216bf48, scope.system=all) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:29.145580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.147071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.163229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 161/310] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:28 2022] GET /identity/v3/role_assignments?user.id=4c5cf2cca21343199b7f0516d216bf48&scope.system=all&role.id=e5378d5e9b8948bf921c2e75bc97b128 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:29.163229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.163923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 150/311] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:29.178091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d347c-fcd9-4a5c-a7dc-aefcc9109561 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.228624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.303285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335bd1e-fb02-4cc8-9555-a9bf2df0fcca None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.324013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d347c-fcd9-4a5c-a7dc-aefcc9109561 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.353190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.354244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.354571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.354745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] PATH_INFO: `/v3/system/users/4adaa81489664dc9be39ed4830804e46/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.356310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4adaa81489664dc9be39ed4830804e46, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.357704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.377304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.390596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.392046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.392524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.393001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.410041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:29.425642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:29.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:29.427391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:29.428132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 151/312] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:29.434003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.441374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 162/313] 10.222.0.43 () {58 vars in 1334 bytes} [Thu Jun 23 10:49:29 2022] PUT /identity/v3/system/users/4adaa81489664dc9be39ed4830804e46/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:29.552072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8e4be-9e91-4eff-a2f1-43ccf8990273 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.592154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.593019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.593409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.593811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.594691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:29.595109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.596648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.611525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 152/314] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:29.613288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.726778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca74815-b1d4-470a-a6d3-95d8b17b4e8f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.739016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.763821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.764897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.765291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.765707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] PATH_INFO: `/v3/users/b9f02a000c504ebd8c07bf8da50ed995` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:29.767081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] RBAC: Authorizing `identity:get_user(user_id=b9f02a000c504ebd8c07bf8da50ed995)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:29.768769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:29.770733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 163/315] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/users/b9f02a000c504ebd8c07bf8da50ed995 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:29.776947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:29.831871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:29.888430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce343329-2b4e-4516-b1e0-126e2161626e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:29.938279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:29.939213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:29.939664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:29.940095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.054578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:30.056409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:30.112674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:30.166614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 153/316] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:30.172197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.282621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d34d9-1c25-411f-ab16-2bf5070cbf34 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.318174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba825f-5cb0-4349-a5dd-af88b86c48d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.319045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba825f-5cb0-4349-a5dd-af88b86c48d3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.319368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba825f-5cb0-4349-a5dd-af88b86c48d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.319649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba825f-5cb0-4349-a5dd-af88b86c48d3 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.320856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba825f-5cb0-4349-a5dd-af88b86c48d3 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:30.322304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba825f-5cb0-4349-a5dd-af88b86c48d3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:30.324667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 164/317] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:30.332299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:30.439169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b005d20-252a-4301-be1b-61d02d295feb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:30.473546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f016-e4da-4cd0-a697-192324518be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:30.474674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f016-e4da-4cd0-a697-192324518be7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.475131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f016-e4da-4cd0-a697-192324518be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.475575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f016-e4da-4cd0-a697-192324518be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.476542 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053f016-e4da-4cd0-a697-192324518be7 None admin] RBAC: Adding query filter params (role.id=e5378d5e9b8948bf921c2e75bc97b128, scope.project.id=3619df4b249349909961aab754e48694, user.id=b9f02a000c504ebd8c07bf8da50ed995) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:30.477012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053f016-e4da-4cd0-a697-192324518be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:30.478570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053f016-e4da-4cd0-a697-192324518be7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:30.485471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 154/318] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:30 2022] GET /identity/v3/role_assignments?user.id=b9f02a000c504ebd8c07bf8da50ed995&scope.project.id=3619df4b249349909961aab754e48694&role.id=e5378d5e9b8948bf921c2e75bc97b128 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:30.943414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2c400-a901-44ce-a79a-bc52739e2569 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.943699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2c400-a901-44ce-a79a-bc52739e2569 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.943980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2c400-a901-44ce-a79a-bc52739e2569 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.945485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 165/319] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:30.951069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013fed-ccf3-469c-8f45-9c43b35e60a6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:30.951393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013fed-ccf3-469c-8f45-9c43b35e60a6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:30.951676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013fed-ccf3-469c-8f45-9c43b35e60a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:30.968009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b2013fed-ccf3-469c-8f45-9c43b35e60a6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:30.969294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b2013fed-ccf3-469c-8f45-9c43b35e60a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:30.978693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.036900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 155/320] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:31.059379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2c400-a901-44ce-a79a-bc52739e2569 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.107441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.164713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2c400-a901-44ce-a79a-bc52739e2569 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.199509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88efff-33a8-4f05-82ee-f139bdf082fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.200220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88efff-33a8-4f05-82ee-f139bdf082fb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.200506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88efff-33a8-4f05-82ee-f139bdf082fb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.200786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88efff-33a8-4f05-82ee-f139bdf082fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.201946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88efff-33a8-4f05-82ee-f139bdf082fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.203420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88efff-33a8-4f05-82ee-f139bdf082fb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.205192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 166/321] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.210437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013fed-ccf3-469c-8f45-9c43b35e60a6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.310569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013fed-ccf3-469c-8f45-9c43b35e60a6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.313599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3f4d-93fd-416c-87f0-b429563ccc44 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.313896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3f4d-93fd-416c-87f0-b429563ccc44 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.314184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3f4d-93fd-416c-87f0-b429563ccc44 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.315719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 167/322] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:31.320556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80530f-5fb2-4830-b908-8b61f550268b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.320851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80530f-5fb2-4830-b908-8b61f550268b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.321135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80530f-5fb2-4830-b908-8b61f550268b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.340037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cc80530f-5fb2-4830-b908-8b61f550268b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:31.341255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cc80530f-5fb2-4830-b908-8b61f550268b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:31.346154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.346827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.347111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.347419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] PATH_INFO: `/v3/groups` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.348119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.349828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.411438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 168/323] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:31.434240 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80530f-5fb2-4830-b908-8b61f550268b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.455675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 156/324] 10.222.0.43 () {60 vars in 1134 bytes} [Thu Jun 23 10:49:31 2022] POST /identity/v3/groups => generated 226 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:31.492384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.552275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80530f-5fb2-4830-b908-8b61f550268b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.589856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57071b17-6738-4a00-8846-6b0b0f7608a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.590677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57071b17-6738-4a00-8846-6b0b0f7608a3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.590958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57071b17-6738-4a00-8846-6b0b0f7608a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.591258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57071b17-6738-4a00-8846-6b0b0f7608a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.603017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57071b17-6738-4a00-8846-6b0b0f7608a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-57071b17-6738-4a00-8846-6b0b0f7608a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:31.611137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:31.612066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:31.613980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 169/325] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:31.620487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.626860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.700308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b56e8-e5cd-4cef-8749-2d48578c58ed None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.700575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b56e8-e5cd-4cef-8749-2d48578c58ed None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.700860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b56e8-e5cd-4cef-8749-2d48578c58ed None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.702381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 170/326] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:31.708749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824dc6-2dbe-44b3-9784-27bb045a7429 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.708974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824dc6-2dbe-44b3-9784-27bb045a7429 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.709251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824dc6-2dbe-44b3-9784-27bb045a7429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.734358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-57824dc6-2dbe-44b3-9784-27bb045a7429 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:31.734358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-57824dc6-2dbe-44b3-9784-27bb045a7429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:31.738217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33019dd9-cad1-436d-93f1-775552192118 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.781219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af89c91-292d-4102-917c-828421d03758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.781897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af89c91-292d-4102-917c-828421d03758 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.782183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af89c91-292d-4102-917c-828421d03758 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.782464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af89c91-292d-4102-917c-828421d03758 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.783263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af89c91-292d-4102-917c-828421d03758 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:31.783581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af89c91-292d-4102-917c-828421d03758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.785399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af89c91-292d-4102-917c-828421d03758 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.794488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 157/327] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:31.800198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af89c91-292d-4102-917c-828421d03758 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.814081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 171/328] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:31.837851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824dc6-2dbe-44b3-9784-27bb045a7429 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:31.874950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.881992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:31.937868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af89c91-292d-4102-917c-828421d03758 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.962909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824dc6-2dbe-44b3-9784-27bb045a7429 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:31.982695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:31.983999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:31.984353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:31.984704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] PATH_INFO: `/v3/users/4c5cf2cca21343199b7f0516d216bf48` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:31.986128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] RBAC: Authorizing `identity:get_user(user_id=4c5cf2cca21343199b7f0516d216bf48)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:31.987859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:31.994255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 158/329] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/users/4c5cf2cca21343199b7f0516d216bf48 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:31.996129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:32.010468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.011711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.012325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.012951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.025162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:32.034118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:32.035071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:32.035933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:32.038733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 172/330] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:32.046073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:32.115324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25432c8-381f-432f-8562-ea3c4bf2393e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:32.168344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58401663-9871-4690-93fb-7991100e275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.169256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401663-9871-4690-93fb-7991100e275a None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.169542 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401663-9871-4690-93fb-7991100e275a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.169827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401663-9871-4690-93fb-7991100e275a None admin] PATH_INFO: `/v3/system/users/4c5cf2cca21343199b7f0516d216bf48/roles/e5378d5e9b8948bf921c2e75bc97b128` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.171359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58401663-9871-4690-93fb-7991100e275a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4c5cf2cca21343199b7f0516d216bf48, role_id=e5378d5e9b8948bf921c2e75bc97b128)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:32.173214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58401663-9871-4690-93fb-7991100e275a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:32.175096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:32.182106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:32.190534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05a1c4-d922-472c-adef-65fc281921ff None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:32.230641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.231286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.231549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.231831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.232653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:32.232968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:32.234439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:32.245383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 173/331] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:32.250683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:32.263882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 159/332] 10.222.0.43 () {58 vars in 1334 bytes} [Thu Jun 23 10:49:31 2022] PUT /identity/v3/system/users/4c5cf2cca21343199b7f0516d216bf48/roles/e5378d5e9b8948bf921c2e75bc97b128 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:32.380012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0510053-f3ed-4490-827e-de4eb52dbc3b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:32.419755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e5e45-423a-4b93-bffe-c23d8050de12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.420656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e5e45-423a-4b93-bffe-c23d8050de12 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.421137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e5e45-423a-4b93-bffe-c23d8050de12 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.421430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e5e45-423a-4b93-bffe-c23d8050de12 None admin] PATH_INFO: `/v3/users/4adaa81489664dc9be39ed4830804e46` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.422731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e5e45-423a-4b93-bffe-c23d8050de12 None admin] RBAC: Authorizing `identity:get_user(user_id=4adaa81489664dc9be39ed4830804e46)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:32.424313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e5e45-423a-4b93-bffe-c23d8050de12 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:32.426437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 174/333] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:32 2022] GET /identity/v3/users/4adaa81489664dc9be39ed4830804e46 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:32.433200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58401663-9871-4690-93fb-7991100e275a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:32.552558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58401663-9871-4690-93fb-7991100e275a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:32.588718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc30381-dc34-4330-adf3-2c8de85d9175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:32.589618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc30381-dc34-4330-adf3-2c8de85d9175 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:32.589840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc30381-dc34-4330-adf3-2c8de85d9175 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:32.590118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc30381-dc34-4330-adf3-2c8de85d9175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:32.591015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc30381-dc34-4330-adf3-2c8de85d9175 None admin] RBAC: Adding query filter params (role.id=adb9ae992e5a4594a8f7510556b4db5f, user.id=4adaa81489664dc9be39ed4830804e46, scope.system=all) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:32.591389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc30381-dc34-4330-adf3-2c8de85d9175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:32.592859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc30381-dc34-4330-adf3-2c8de85d9175 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:32.605519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 160/334] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:32 2022] GET /identity/v3/role_assignments?user.id=4adaa81489664dc9be39ed4830804e46&scope.system=all&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:33.820817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58fe8c-9a81-43c2-82ac-35b2ca47efdc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:33.821359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58fe8c-9a81-43c2-82ac-35b2ca47efdc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:33.821452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58fe8c-9a81-43c2-82ac-35b2ca47efdc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:33.823036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 175/335] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:33.828655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1b06-80d8-49d9-a7e4-73dd5b0eb9a7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:33.829079 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1b06-80d8-49d9-a7e4-73dd5b0eb9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:33.829279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1b06-80d8-49d9-a7e4-73dd5b0eb9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:33.853260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a3fc1b06-80d8-49d9-a7e4-73dd5b0eb9a7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:33.862544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a3fc1b06-80d8-49d9-a7e4-73dd5b0eb9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:33.929623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:33.952389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 161/336] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:33.992687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58fe8c-9a81-43c2-82ac-35b2ca47efdc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:34.032066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:34.127978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58fe8c-9a81-43c2-82ac-35b2ca47efdc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:34.172855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:34.173663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:34.173951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:34.174230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:34.191634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:34.205677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:34.207879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:34.209765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 176/337] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:34.217341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1b06-80d8-49d9-a7e4-73dd5b0eb9a7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:34.334105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1b06-80d8-49d9-a7e4-73dd5b0eb9a7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:34.374743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd903ee-9af8-4c93-981d-1382a262691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:34.375460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd903ee-9af8-4c93-981d-1382a262691b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:34.375744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd903ee-9af8-4c93-981d-1382a262691b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:34.376026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd903ee-9af8-4c93-981d-1382a262691b None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:34.376786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd903ee-9af8-4c93-981d-1382a262691b None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:34.377096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd903ee-9af8-4c93-981d-1382a262691b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:34.378535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd903ee-9af8-4c93-981d-1382a262691b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:34.388327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 162/338] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:34.393892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:34.515808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:34.526493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a30dd2-983a-4732-ba59-8d252e8f39ae None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:34.565293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:34.565950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:34.566238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:34.566520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:34.592897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d601f-0f7d-4d2b-b54c-03d7e9fd6706 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:34.592897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d601f-0f7d-4d2b-b54c-03d7e9fd6706 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:34.592897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d601f-0f7d-4d2b-b54c-03d7e9fd6706 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:34.596176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 163/339] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:34.599030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fca4b5-aeb7-47b3-9818-9007b943b908 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:34.599573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fca4b5-aeb7-47b3-9818-9007b943b908 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:34.600049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fca4b5-aeb7-47b3-9818-9007b943b908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:34.643120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-59fca4b5-aeb7-47b3-9818-9007b943b908 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:34.643655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:34.644888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-59fca4b5-aeb7-47b3-9818-9007b943b908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:34.645168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:34.723516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:34.747206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:34.757199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 177/340] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:34.771832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 164/341] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:34.778720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:34.816663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fca4b5-aeb7-47b3-9818-9007b943b908 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:34.932752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95421a-5ec7-4b76-aa14-bb7385f00e0e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.010810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.012169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.012646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.013099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.013373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fca4b5-aeb7-47b3-9818-9007b943b908 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.015069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.016743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.019102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 178/342] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:34 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:35.024567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.107906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef9865-cc76-4bce-9ada-c64db3095ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.108737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef9865-cc76-4bce-9ada-c64db3095ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.109023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef9865-cc76-4bce-9ada-c64db3095ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.109302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef9865-cc76-4bce-9ada-c64db3095ed8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.132286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef9865-cc76-4bce-9ada-c64db3095ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:35.135573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0eef9865-cc76-4bce-9ada-c64db3095ed8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:35.140706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:35.141609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:35.142418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:35.147589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 165/343] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:35.148844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eaab3-3509-4dba-a6fb-f6a2b0df49ce None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.149133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eaab3-3509-4dba-a6fb-f6a2b0df49ce None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.149412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8eaab3-3509-4dba-a6fb-f6a2b0df49ce None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.151032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 166/344] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:35.156517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eaab3-3509-4dba-a6fb-f6a2b0df49ce None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.206621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388699b-09f2-46ee-bdd7-5a9bbbb7ec4c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.230632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.246684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67fc7d-df3c-449a-bc50-0db9aa029beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.249258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67fc7d-df3c-449a-bc50-0db9aa029beb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.249965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67fc7d-df3c-449a-bc50-0db9aa029beb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.250417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67fc7d-df3c-449a-bc50-0db9aa029beb None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.257766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67fc7d-df3c-449a-bc50-0db9aa029beb None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.259457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67fc7d-df3c-449a-bc50-0db9aa029beb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.267288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 179/345] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:35.269593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d74e96-2a5c-42b0-99a1-ab4254150ae5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.270044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d74e96-2a5c-42b0-99a1-ab4254150ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.270716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d74e96-2a5c-42b0-99a1-ab4254150ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.289575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f6d74e96-2a5c-42b0-99a1-ab4254150ae5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:35.290954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f6d74e96-2a5c-42b0-99a1-ab4254150ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:35.363370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8eaab3-3509-4dba-a6fb-f6a2b0df49ce None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.376204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 180/346] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:35.377375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d74e96-2a5c-42b0-99a1-ab4254150ae5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.425037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.425842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.426133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.426415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.427240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:35.427516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.430910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.442037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 167/347] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:35.442890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.447624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.511597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d74e96-2a5c-42b0-99a1-ab4254150ae5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.563026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bb61c-87d4-4171-8f90-39bd03865d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.563896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb61c-87d4-4171-8f90-39bd03865d78 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.564180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb61c-87d4-4171-8f90-39bd03865d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.564455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb61c-87d4-4171-8f90-39bd03865d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.565301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bb61c-87d4-4171-8f90-39bd03865d78 None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=2edc2b4730164d7195bc4fc982443c5b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:35.565614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bb61c-87d4-4171-8f90-39bd03865d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.567065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bb61c-87d4-4171-8f90-39bd03865d78 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:35.572873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.575898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd972-70d9-47d6-a3bc-5237a87fa8dd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.582875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 181/348] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:35.582875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0b53-5165-4e5f-b851-6171510232ac None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.582875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0b53-5165-4e5f-b851-6171510232ac None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.582875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0b53-5165-4e5f-b851-6171510232ac None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.582875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 182/349] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:35.585759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c0b53-5165-4e5f-b851-6171510232ac None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.653339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843d081-9e13-49c0-a488-9e63a3355f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.654160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843d081-9e13-49c0-a488-9e63a3355f35 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.654446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843d081-9e13-49c0-a488-9e63a3355f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.654721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843d081-9e13-49c0-a488-9e63a3355f35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.683912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843d081-9e13-49c0-a488-9e63a3355f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e843d081-9e13-49c0-a488-9e63a3355f35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:35.696856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:35.700508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:35.701335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 168/350] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:35.704278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecc496-95f0-454f-ad7e-1606a0d809e5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.704278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecc496-95f0-454f-ad7e-1606a0d809e5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.704278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecc496-95f0-454f-ad7e-1606a0d809e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.740023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-adecc496-95f0-454f-ad7e-1606a0d809e5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:35.743802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-adecc496-95f0-454f-ad7e-1606a0d809e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:35.802722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c0b53-5165-4e5f-b851-6171510232ac None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:35.815268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:35.842011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:35.842487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:35.842773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:35.843049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:35.868565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 169/351] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:35.870002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecc496-95f0-454f-ad7e-1606a0d809e5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:35.942881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:35.944535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.035330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.053742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecc496-95f0-454f-ad7e-1606a0d809e5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.130969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.131702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.131990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.132266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.133034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:36.133348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.134746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.156023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 183/352] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:36.156886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.167346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 170/353] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.168245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.182669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.307781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ea9f1-9ceb-48aa-b2f7-74ffd88633b2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.323579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c4184-3e97-4e89-8732-b0d5466d78ad None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.350473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f858ebc-2a64-441d-91ae-d4c8b1f73ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.351762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f858ebc-2a64-441d-91ae-d4c8b1f73ace None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.352277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f858ebc-2a64-441d-91ae-d4c8b1f73ace None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.352759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f858ebc-2a64-441d-91ae-d4c8b1f73ace None admin] PATH_INFO: `/v3/roles/member` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.365454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f858ebc-2a64-441d-91ae-d4c8b1f73ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:36.367066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.367961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.368243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.368516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.369772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.372962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1f858ebc-2a64-441d-91ae-d4c8b1f73ace None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:36.376896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:36.377766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:36.378582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:36.379204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 171/354] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.379204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.381348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 184/355] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:36.382864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1630b02-d916-4d4e-b623-1c823c97bb7a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.383386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1630b02-d916-4d4e-b623-1c823c97bb7a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.383899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1630b02-d916-4d4e-b623-1c823c97bb7a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.385396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 185/356] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:36.386309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1630b02-d916-4d4e-b623-1c823c97bb7a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.409433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.472128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.510417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57447e7f-adb3-4674-943b-20b89a54ac84 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.510624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1630b02-d916-4d4e-b623-1c823c97bb7a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.552482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.553739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.554170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200796c6-5000-4469-81a8-17d4802651ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.554466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.554934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.555035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200796c6-5000-4469-81a8-17d4802651ec None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.555352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200796c6-5000-4469-81a8-17d4802651ec None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.555626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200796c6-5000-4469-81a8-17d4802651ec None admin] PATH_INFO: `/v3/users/4c5cf2cca21343199b7f0516d216bf48` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.557573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.557747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200796c6-5000-4469-81a8-17d4802651ec None admin] RBAC: Authorizing `identity:get_user(user_id=4c5cf2cca21343199b7f0516d216bf48)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.559235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200796c6-5000-4469-81a8-17d4802651ec None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.559693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.560845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 172/357] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/users/4c5cf2cca21343199b7f0516d216bf48 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.561784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 186/358] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.562735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.566123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a8539-f5cb-41bc-b72b-0c974408f7ce None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.566440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a8539-f5cb-41bc-b72b-0c974408f7ce None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.566726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a8539-f5cb-41bc-b72b-0c974408f7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.585966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-014a8539-f5cb-41bc-b72b-0c974408f7ce None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:36.587149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-014a8539-f5cb-41bc-b72b-0c974408f7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:36.675472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 173/359] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:36.676391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a8539-f5cb-41bc-b72b-0c974408f7ce None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.677866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464de7-9025-40d7-89d0-7b1098465424 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.695840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.718042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.718778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.720623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.720623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.720623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:36.720623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.721946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.732520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 187/360] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.732876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:36.735279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.809857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a8539-f5cb-41bc-b72b-0c974408f7ce None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.854294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.855286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.855396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.858108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.858222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336d91-4ad8-41c0-a803-8cb025c0376a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:36.858302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] RBAC: Adding query filter params (role.id=e5378d5e9b8948bf921c2e75bc97b128, user.id=4c5cf2cca21343199b7f0516d216bf48, scope.system=all) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:36.858302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.858302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.874519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 174/361] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/role_assignments?user.id=4c5cf2cca21343199b7f0516d216bf48&scope.system=all&role.id=e5378d5e9b8948bf921c2e75bc97b128 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.875399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.921482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f989a-86b5-4808-8f20-7bae17890430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:36.922661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f989a-86b5-4808-8f20-7bae17890430 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.923150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f989a-86b5-4808-8f20-7bae17890430 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.923668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f989a-86b5-4808-8f20-7bae17890430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.924732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f989a-86b5-4808-8f20-7bae17890430 None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=2edc2b4730164d7195bc4fc982443c5b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:36.925261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f989a-86b5-4808-8f20-7bae17890430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:36.927021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f989a-86b5-4808-8f20-7bae17890430 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:36.935009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 188/362] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:36.936780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379af24-ec4f-4d66-922c-9de35f1af70d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:36.937289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379af24-ec4f-4d66-922c-9de35f1af70d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:36.937774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379af24-ec4f-4d66-922c-9de35f1af70d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:36.939841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 189/363] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:36.944278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379af24-ec4f-4d66-922c-9de35f1af70d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:36.997177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.019635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.035097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede18e3-31c1-417a-a1d2-36e74ff8826d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.069833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379af24-ec4f-4d66-922c-9de35f1af70d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.097633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0d772-3441-41fb-ad2b-680b3672062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.098476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d772-3441-41fb-ad2b-680b3672062f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.099026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d772-3441-41fb-ad2b-680b3672062f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.099344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d772-3441-41fb-ad2b-680b3672062f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.112764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.113276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0d772-3441-41fb-ad2b-680b3672062f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:37.114024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.114525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.114997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-6ef0d772-3441-41fb-ad2b-680b3672062f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:37.122629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:37.123647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:37.124442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 175/364] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:37.125590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c30af7-28e7-477b-bdf0-329a367c28a3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.125590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c30af7-28e7-477b-bdf0-329a367c28a3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.125848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c30af7-28e7-477b-bdf0-329a367c28a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.153422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-75c30af7-28e7-477b-bdf0-329a367c28a3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:37.154791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-75c30af7-28e7-477b-bdf0-329a367c28a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:37.197914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:37.199840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.240016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 176/365] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:37.241327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c30af7-28e7-477b-bdf0-329a367c28a3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.269578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.309710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 190/366] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:37.310872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.321441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.361313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c30af7-28e7-477b-bdf0-329a367c28a3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.399328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.400376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.401456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.402059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.403032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:37.403642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:37.405282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.414442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 177/367] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:37.415491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.420194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30b171-ff3d-43ec-ab53-d940988694d2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.456464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.457398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.457748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.458093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.471313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:37.480785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:37.482244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:37.483055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 191/368] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:37.483756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.512286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.539560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac678-dc66-467b-bebb-3237de4ce2e6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.572807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.579311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.580436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.580904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.581388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.583288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:37.590803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.590803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 178/369] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:37.591398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.600687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d503c9f-0ecc-4679-ac92-582f6bf5a064 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.637002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.637783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.638135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.638483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.713541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bff25c-41d9-46d7-a8fa-48066e6daa58 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.726397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:37.728319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.756876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.757657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.758022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.758375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.758859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.759363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:37.760265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:37.761396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.770824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 179/370] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:37.771833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.839998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:37.858637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 192/371] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:37.859549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:37.893124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda9698-3962-40b1-b818-2205c951e283 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:37.937439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:37.938678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:37.939201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:37.940529 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:37.942242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:37.944900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:37.948356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 180/372] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:37.949355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.012331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d113c-3391-4b33-a683-c043eebcd27d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:38.042201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:38.054496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50503d-fe6e-4e2f-a7b6-826ad58083f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:38.055159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50503d-fe6e-4e2f-a7b6-826ad58083f3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.055476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50503d-fe6e-4e2f-a7b6-826ad58083f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.058883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50503d-fe6e-4e2f-a7b6-826ad58083f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.083363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146f02-930c-4452-b7ed-4a612aafc6b7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:38.116218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:38.128902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:38.130246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.137752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.137752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.137752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:38.137752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:38.138350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 181/373] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:38.139538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.141409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50503d-fe6e-4e2f-a7b6-826ad58083f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:38.142880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50503d-fe6e-4e2f-a7b6-826ad58083f3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:38.266601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096feea-a3bd-46b3-9ed5-2ebfb923cd2c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:38.271976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 193/374] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:38.273489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf11413-3fe6-4899-ac92-5a1ecd467cf8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.273787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf11413-3fe6-4899-ac92-5a1ecd467cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.274074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf11413-3fe6-4899-ac92-5a1ecd467cf8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.279720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 194/375] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:38.280582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf11413-3fe6-4899-ac92-5a1ecd467cf8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.306112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:38.308220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.308220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.308220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.309256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=3619df4b249349909961aab754e48694) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:38.310272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:38.311242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:38.317297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:38.323587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 182/376] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=3619df4b249349909961aab754e48694&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:38.324207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.400930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:38.424984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf11413-3fe6-4899-ac92-5a1ecd467cf8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:38.463822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17d9c2-f314-43ec-ad5e-0e9819487652 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:38.473047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da91104-a61c-4d94-aa74-085563669e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:38.474312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91104-a61c-4d94-aa74-085563669e93 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.474665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91104-a61c-4d94-aa74-085563669e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.475405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91104-a61c-4d94-aa74-085563669e93 None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.476783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da91104-a61c-4d94-aa74-085563669e93 None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:38.479090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da91104-a61c-4d94-aa74-085563669e93 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:38.483297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 195/377] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:38.484691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309f20-ac22-4495-8c96-1487f899158e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.484983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309f20-ac22-4495-8c96-1487f899158e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.485263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309f20-ac22-4495-8c96-1487f899158e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.514718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-48309f20-ac22-4495-8c96-1487f899158e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:38.516283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-48309f20-ac22-4495-8c96-1487f899158e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:38.517090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:38.518211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.518740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.519021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] PATH_INFO: `/v3/projects/d945ea14d2824c65a22104765fba4b15` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.520310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] RBAC: Authorizing `identity:get_project(project_id=d945ea14d2824c65a22104765fba4b15)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:38.521860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:38.523774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 183/378] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/projects/d945ea14d2824c65a22104765fba4b15 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:38.524377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.607715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 196/379] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:38.612085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48309f20-ac22-4495-8c96-1487f899158e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.660927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:38.670700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af759-f756-448c-81a6-6187d14c8f4a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:38.706940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:38.717162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:38.718026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.718382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.718812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.720033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=3619df4b249349909961aab754e48694) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:38.720322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:38.722004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:38.730437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 184/380] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=3619df4b249349909961aab754e48694&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:38.731440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.781299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48309f20-ac22-4495-8c96-1487f899158e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:38.870722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:38.871814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:38.872207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:38.872590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] PATH_INFO: `/v3/groups/91837d0847c44e65a535025a25ef1e16` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:38.887925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] RBAC: Authorizing `identity:get_group(group_id=91837d0847c44e65a535025a25ef1e16)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:38.889706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:38.893820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 197/381] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/groups/91837d0847c44e65a535025a25ef1e16 => generated 229 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:38.897227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:38.947103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5475792-dfb5-4e27-8049-173972bc7dca None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.000672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.001494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.001934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.002291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.017235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:39.031212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:39.032166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:39.032946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:39.034029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 185/382] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:39.038867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.043679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.053356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfc69a-4960-49b6-9795-dbd2252a43ca None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.101492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.102165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.102456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.102740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.104202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] RBAC: Adding query filter params (group.id=91837d0847c44e65a535025a25ef1e16, role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=d945ea14d2824c65a22104765fba4b15) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:39.104563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.106046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.116714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 198/383] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:38 2022] GET /identity/v3/role_assignments?group.id=91837d0847c44e65a535025a25ef1e16&scope.project.id=d945ea14d2824c65a22104765fba4b15&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:39.151518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.215540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7534e-9d9f-4abd-8b12-4c20e8bb6e18 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.267790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.268726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.269156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.269605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.270511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:39.270963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.272567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.287652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 186/384] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:39.294846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.455493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df971e-d554-4ff9-a822-6998fe6891fd None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.497351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd4aad-2e9d-4153-bb57-f8c38537d60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.498140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd4aad-2e9d-4153-bb57-f8c38537d60e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.498422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd4aad-2e9d-4153-bb57-f8c38537d60e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.498700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd4aad-2e9d-4153-bb57-f8c38537d60e None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.499965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd4aad-2e9d-4153-bb57-f8c38537d60e None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.501448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd4aad-2e9d-4153-bb57-f8c38537d60e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.503410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 199/385] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:39.508595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.541461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87794b-7c7c-4414-acf8-2b15d803f4f3 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.541911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87794b-7c7c-4414-acf8-2b15d803f4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.542096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87794b-7c7c-4414-acf8-2b15d803f4f3 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.543501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 200/386] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:39.559361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34197d-ca38-4c33-9b93-f74de8eca4e0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.559361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34197d-ca38-4c33-9b93-f74de8eca4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.559361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34197d-ca38-4c33-9b93-f74de8eca4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.586990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3d34197d-ca38-4c33-9b93-f74de8eca4e0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:39.589481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3d34197d-ca38-4c33-9b93-f74de8eca4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:39.611591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.718561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4051d-e018-4f99-932d-ab56e7f15ffb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.721619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 201/387] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:39.747637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:39.751190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34197d-ca38-4c33-9b93-f74de8eca4e0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.787425 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.788312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.788597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.788877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.790443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:39.792411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:39.794150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 187/388] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:39.800089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.895617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34197d-ca38-4c33-9b93-f74de8eca4e0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:39.935168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:39.936030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:39.936319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:39.936597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:39.949168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:39.957341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:39.958205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:39.959002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 202/389] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:39.965527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:39.995570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.064863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c24fa-3747-479c-819b-5e386c384699 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.126968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97103e-31cf-4ca0-a7ea-46bda1fd31ca None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.156042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.156965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.157258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.157546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b/groups/c84c932d1f964502b210a1b802366260/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.163976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] RBAC: Authorizing `identity:create_grant(project_id=2edc2b4730164d7195bc4fc982443c5b, group_id=c84c932d1f964502b210a1b802366260, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.165441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.166393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.167078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.167586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.168059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.168982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:40.169300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.170755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.180040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 203/390] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:40.185707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.262070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.272023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 188/391] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:39 2022] PUT /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b/groups/c84c932d1f964502b210a1b802366260/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:40.298402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0e25-0023-4c43-884f-696e86b5e6e2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.337877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.338682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.338970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.339285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.340771 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.342259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.344403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 204/392] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:40.352326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.423286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.497340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097493c6-70de-41ba-ab78-9cd14c593d43 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.535930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0dfa5-12bb-429c-924f-e75e0db3bc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.536810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0dfa5-12bb-429c-924f-e75e0db3bc08 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.537099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0dfa5-12bb-429c-924f-e75e0db3bc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.537384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0dfa5-12bb-429c-924f-e75e0db3bc08 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.539119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0dfa5-12bb-429c-924f-e75e0db3bc08 None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.540588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0dfa5-12bb-429c-924f-e75e0db3bc08 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.543658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 189/393] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:40 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:40.548425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:40.690739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f381d-d4eb-49bd-960f-e9e15fd5ff2a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:40.728695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671626fe-a03b-45f0-baec-18866f677a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:40.729625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671626fe-a03b-45f0-baec-18866f677a83 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.729911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671626fe-a03b-45f0-baec-18866f677a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.730193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671626fe-a03b-45f0-baec-18866f677a83 None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b/groups/c84c932d1f964502b210a1b802366260/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.732474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671626fe-a03b-45f0-baec-18866f677a83 None admin] RBAC: Authorizing `identity:create_grant(project_id=2edc2b4730164d7195bc4fc982443c5b, group_id=c84c932d1f964502b210a1b802366260, role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:40.733944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671626fe-a03b-45f0-baec-18866f677a83 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:40.800421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:40.806179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 205/394] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:40 2022] PUT /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b/groups/c84c932d1f964502b210a1b802366260/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:40.949629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fb841-c53c-48a7-a3a8-bc181ffc3541 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.949629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fb841-c53c-48a7-a3a8-bc181ffc3541 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.949629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fb841-c53c-48a7-a3a8-bc181ffc3541 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.950470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 190/395] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:40.955969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fd544-036f-41fa-9b90-870c3264c44a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:40.956548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fd544-036f-41fa-9b90-870c3264c44a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:40.957032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fd544-036f-41fa-9b90-870c3264c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:40.974747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b77fd544-036f-41fa-9b90-870c3264c44a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:40.976322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b77fd544-036f-41fa-9b90-870c3264c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:41.054682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 206/396] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:41.077347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fb841-c53c-48a7-a3a8-bc181ffc3541 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.195862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fb841-c53c-48a7-a3a8-bc181ffc3541 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.214123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.241826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.242633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.242919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.243228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.256056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:41.267720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:41.268632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:41.269517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:41.270121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 191/397] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:41.276233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fd544-036f-41fa-9b90-870c3264c44a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.386968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fd544-036f-41fa-9b90-870c3264c44a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.415840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.424374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c10ca5-27d1-4007-83b3-2be97aea0f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.425065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c10ca5-27d1-4007-83b3-2be97aea0f71 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.425355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c10ca5-27d1-4007-83b3-2be97aea0f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.425636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c10ca5-27d1-4007-83b3-2be97aea0f71 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.432468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c10ca5-27d1-4007-83b3-2be97aea0f71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:41.432468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c10ca5-27d1-4007-83b3-2be97aea0f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.432468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c10ca5-27d1-4007-83b3-2be97aea0f71 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.438267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 207/398] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:41.443774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.465157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6d0f-2369-432e-b819-06bb275293c5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.465432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6d0f-2369-432e-b819-06bb275293c5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.465639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6d0f-2369-432e-b819-06bb275293c5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.467285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 208/399] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:41.473780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaa3c1-7486-44f1-af85-c2c05e41e4b4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.474070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaa3c1-7486-44f1-af85-c2c05e41e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.474353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aaa3c1-7486-44f1-af85-c2c05e41e4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.499055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-94aaa3c1-7486-44f1-af85-c2c05e41e4b4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:41.500303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-94aaa3c1-7486-44f1-af85-c2c05e41e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:41.638064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 209/400] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:41.646939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d35c80-963d-44a4-9fa4-e189dabcdba8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.683085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.684045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.684331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.684608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.686575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaa3c1-7486-44f1-af85-c2c05e41e4b4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.695809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.700734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.702659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 192/401] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:41.708785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.772528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.856889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aaa3c1-7486-44f1-af85-c2c05e41e4b4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.891954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:41.895484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a492b-4d09-4c8d-bb04-ced8018e6b45 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:41.919828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40dd98-0b53-47e4-bd47-da79d8ef6717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.920970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40dd98-0b53-47e4-bd47-da79d8ef6717 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.921470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40dd98-0b53-47e4-bd47-da79d8ef6717 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.921943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40dd98-0b53-47e4-bd47-da79d8ef6717 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.934204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40dd98-0b53-47e4-bd47-da79d8ef6717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:41.935160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3273bf5-480e-4bee-90c3-e01bf1bd946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:41.936060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3273bf5-480e-4bee-90c3-e01bf1bd946b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.936356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3273bf5-480e-4bee-90c3-e01bf1bd946b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.936638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3273bf5-480e-4bee-90c3-e01bf1bd946b None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.937848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3273bf5-480e-4bee-90c3-e01bf1bd946b None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:41.939563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3273bf5-480e-4bee-90c3-e01bf1bd946b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:41.945609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 193/402] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:41.945609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2745e4-4e64-4f4f-bffc-cb7fad0e1664 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.945609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2745e4-4e64-4f4f-bffc-cb7fad0e1664 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.945609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2745e4-4e64-4f4f-bffc-cb7fad0e1664 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.945609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 194/403] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:41.949406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2745e4-4e64-4f4f-bffc-cb7fad0e1664 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-cf40dd98-0b53-47e4-bd47-da79d8ef6717 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:41.950774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:41.951696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:41.952718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:41.955067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 210/404] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:41.958977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116927ee-0461-40ff-893f-ec66be38e72d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:41.959626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116927ee-0461-40ff-893f-ec66be38e72d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:41.963478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116927ee-0461-40ff-893f-ec66be38e72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:41.981709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-116927ee-0461-40ff-893f-ec66be38e72d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:41.983036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-116927ee-0461-40ff-893f-ec66be38e72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:42.067247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 211/405] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:42.068301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116927ee-0461-40ff-893f-ec66be38e72d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.080352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2745e4-4e64-4f4f-bffc-cb7fad0e1664 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.083683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.122762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.124109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.124466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.124836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694/groups/c84c932d1f964502b210a1b802366260/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.126927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] RBAC: Authorizing `identity:create_grant(project_id=3619df4b249349909961aab754e48694, group_id=c84c932d1f964502b210a1b802366260, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.128456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.194195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116927ee-0461-40ff-893f-ec66be38e72d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.211876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 195/406] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:41 2022] PUT /identity/v3/projects/3619df4b249349909961aab754e48694/groups/c84c932d1f964502b210a1b802366260/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:42.213113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.215609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.258895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.261832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.262379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.262882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.263903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:42.264460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.266904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.281270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 212/407] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:42.286276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.437667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11401f67-c70c-4a6f-a0d1-4ff6cb10a3da None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.471966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252213e-56f4-491c-8e6a-9c127f12aa69 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.487117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.488208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.488499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.489266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.504199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:42.511360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.512292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.512707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.512991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:42.513174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:42.514039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:42.515291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:42.516087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.516158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 196/408] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:42.520056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.520545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.522460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 213/409] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:42.527985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.552580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.584944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.687098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce91cd-2e1c-4d0c-bdbd-9e736f627672 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.690572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab2cc0-0211-42fb-b77d-7a06befe079c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.733445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.734123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.734409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.734687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.735471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:42.735783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.737164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.746184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 197/410] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:42.751469 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.752338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.752620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.752897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.754088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.755548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.761978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 214/411] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:42.763666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.767755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:42.938812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0f812-b32e-41db-ab81-ddc3935e5664 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.939972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.952243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:42.955670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27282891-5043-4f77-9d44-1d9826c09e13 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:42.983930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7041c7-ddcb-494e-8b46-93cef705b92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:42.985129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7041c7-ddcb-494e-8b46-93cef705b92e None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:42.985418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7041c7-ddcb-494e-8b46-93cef705b92e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:42.985697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7041c7-ddcb-494e-8b46-93cef705b92e None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694/groups/c84c932d1f964502b210a1b802366260/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:42.989599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7041c7-ddcb-494e-8b46-93cef705b92e None admin] RBAC: Authorizing `identity:create_grant(project_id=3619df4b249349909961aab754e48694, group_id=c84c932d1f964502b210a1b802366260, role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:42.991867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7041c7-ddcb-494e-8b46-93cef705b92e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:42.994996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b7ffd-e9f3-4fad-87fc-e0cbc32e6d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.000923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7ffd-e9f3-4fad-87fc-e0cbc32e6d00 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.000923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7ffd-e9f3-4fad-87fc-e0cbc32e6d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.000923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b7ffd-e9f3-4fad-87fc-e0cbc32e6d00 None admin] PATH_INFO: `/v3/groups/91837d0847c44e65a535025a25ef1e16` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.000923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b7ffd-e9f3-4fad-87fc-e0cbc32e6d00 None admin] RBAC: Authorizing `identity:get_group(group_id=91837d0847c44e65a535025a25ef1e16)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.000923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b7ffd-e9f3-4fad-87fc-e0cbc32e6d00 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.001434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 198/412] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:42 2022] GET /identity/v3/groups/91837d0847c44e65a535025a25ef1e16 => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:43.002630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d041-d206-4ac4-b177-6e6146a374f5 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.002923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d041-d206-4ac4-b177-6e6146a374f5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.003357 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d041-d206-4ac4-b177-6e6146a374f5 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.004584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 199/413] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:43.006520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d041-d206-4ac4-b177-6e6146a374f5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.091043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 215/414] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:42 2022] PUT /identity/v3/projects/3619df4b249349909961aab754e48694/groups/c84c932d1f964502b210a1b802366260/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:49:43.092407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2aa5-718f-4bd1-8a45-87d45609b4e3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.092701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2aa5-718f-4bd1-8a45-87d45609b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.092981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2aa5-718f-4bd1-8a45-87d45609b4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.116843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-058a2aa5-718f-4bd1-8a45-87d45609b4e3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:43.119860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-058a2aa5-718f-4bd1-8a45-87d45609b4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:43.132849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d041-d206-4ac4-b177-6e6146a374f5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.209246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 216/415] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:43.216098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.217095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.217441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.217812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] PATH_INFO: `/v3/projects/d945ea14d2824c65a22104765fba4b15` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.221716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] RBAC: Authorizing `identity:get_project(project_id=d945ea14d2824c65a22104765fba4b15)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.223234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.225611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 200/416] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/projects/d945ea14d2824c65a22104765fba4b15 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:43.231572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2aa5-718f-4bd1-8a45-87d45609b4e3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.264839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.303805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.368479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.374985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2aa5-718f-4bd1-8a45-87d45609b4e3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.391196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51f8c2-5ef7-41cb-b8a3-096c83910fbb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.428251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3598f3-0d9a-41a5-bd9a-1405c58c3e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.430128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3598f3-0d9a-41a5-bd9a-1405c58c3e54 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.430567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3598f3-0d9a-41a5-bd9a-1405c58c3e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.431120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3598f3-0d9a-41a5-bd9a-1405c58c3e54 None admin] PATH_INFO: `/v3/projects/d945ea14d2824c65a22104765fba4b15/groups/91837d0847c44e65a535025a25ef1e16/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.436594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3598f3-0d9a-41a5-bd9a-1405c58c3e54 None admin] RBAC: Authorizing `identity:create_grant(project_id=d945ea14d2824c65a22104765fba4b15, group_id=91837d0847c44e65a535025a25ef1e16, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.437992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3598f3-0d9a-41a5-bd9a-1405c58c3e54 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.439919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.440869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.441255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.441638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.456260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:43.466676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:43.467741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:43.468613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 201/417] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:43.471365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.514935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 217/418] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 10:49:43 2022] PUT /identity/v3/projects/d945ea14d2824c65a22104765fba4b15/groups/91837d0847c44e65a535025a25ef1e16/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:49:43.554299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08bbc3-c310-4b81-8072-4dc6e337f0d5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.554660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08bbc3-c310-4b81-8072-4dc6e337f0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.554889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08bbc3-c310-4b81-8072-4dc6e337f0d5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.563315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 218/419] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:43.567602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767b38a-d12e-4516-bf8b-b93598e2ed7e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.567900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767b38a-d12e-4516-bf8b-b93598e2ed7e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.568187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767b38a-d12e-4516-bf8b-b93598e2ed7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.580178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.601043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495a293-3264-4aa0-a3d1-9f4410d80f93 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.616415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2767b38a-d12e-4516-bf8b-b93598e2ed7e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:43.617684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2767b38a-d12e-4516-bf8b-b93598e2ed7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:43.642744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.644003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.644597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.645162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.646116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:43.646615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.648386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:43.662587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 202/420] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:43.668242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.753040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 219/421] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:43.781964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767b38a-d12e-4516-bf8b-b93598e2ed7e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:43.825448 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef434baf-a114-42fa-a438-df6e9b06ee45 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.872897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:43.875029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:43.875583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:43.876368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:43.899972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.904537 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:43.955952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767b38a-d12e-4516-bf8b-b93598e2ed7e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:43.986452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:43.987952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.014036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.014769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.015054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.015355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.035255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:44.048635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:44.049581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:44.050386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:44.061167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 220/422] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:44.065648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.176251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 203/423] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:44.185930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.224959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e69984-3c03-4788-bf77-5411f0b44f99 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.277229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.284166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.284853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.285146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.285425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.286185 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:44.286496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.287907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.302761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 221/424] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:44.309676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.332772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.384724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb511eb7-c337-4409-9c00-7ad2c20698a0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.448487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.449364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.449656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.449937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.452330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.453797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.455718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 204/425] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:44.462327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.496981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd59f1-0635-4b9d-8bb4-50b1a64ee6ad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.558088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9650e5f-8020-4aa2-bf8a-1bfae797ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.558671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650e5f-8020-4aa2-bf8a-1bfae797ba63 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.558963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650e5f-8020-4aa2-bf8a-1bfae797ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.559306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650e5f-8020-4aa2-bf8a-1bfae797ba63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.653492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe5783-356b-497f-9687-e8790e84993f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.679234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.692894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9650e5f-8020-4aa2-bf8a-1bfae797ba63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.694360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9650e5f-8020-4aa2-bf8a-1bfae797ba63 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.708279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:44.720809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a8982-4f70-44fb-a0cb-648928c9b3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.722512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a8982-4f70-44fb-a0cb-648928c9b3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.723108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a8982-4f70-44fb-a0cb-648928c9b3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.724084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a8982-4f70-44fb-a0cb-648928c9b3e9 None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.726180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a8982-4f70-44fb-a0cb-648928c9b3e9 None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.730914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a8982-4f70-44fb-a0cb-648928c9b3e9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.731447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 205/426] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:44.733404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cd673-e16d-4e55-8cfa-01098630f3cb None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.733789 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cd673-e16d-4e55-8cfa-01098630f3cb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.734167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cd673-e16d-4e55-8cfa-01098630f3cb None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.739563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 206/427] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:44.740496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cd673-e16d-4e55-8cfa-01098630f3cb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.823521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 222/428] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:44.825966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f05f-8f33-4d11-ad44-a5af8abf497e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.826264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f05f-8f33-4d11-ad44-a5af8abf497e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.826551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f05f-8f33-4d11-ad44-a5af8abf497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.846515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2480f05f-8f33-4d11-ad44-a5af8abf497e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:44.848086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2480f05f-8f33-4d11-ad44-a5af8abf497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:44.894162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cd673-e16d-4e55-8cfa-01098630f3cb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:44.935133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 223/429] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:44.935932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480f05f-8f33-4d11-ad44-a5af8abf497e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:44.942154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:44.942828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:44.943119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:44.943428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:44.944271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=2edc2b4730164d7195bc4fc982443c5b) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:44.944596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:44.946036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:44.957028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 207/430] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:44.964497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.007610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.073488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480f05f-8f33-4d11-ad44-a5af8abf497e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.105372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.114692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.115938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.119496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.119496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] PATH_INFO: `/v3/projects/2edc2b4730164d7195bc4fc982443c5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.119496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] RBAC: Authorizing `identity:get_project(project_id=2edc2b4730164d7195bc4fc982443c5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.121414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.123804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 224/431] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/projects/2edc2b4730164d7195bc4fc982443c5b => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:45.131220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.214608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa96a5b-4066-4a5a-b65d-13a50af1c0c6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.258346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.260435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.260867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.261153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.277322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:45.282146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ddd8d-1119-4e66-8657-55f6edf9d5ce None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:45.287369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:45.288270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:45.289078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 208/432] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:45.294487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.298662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.330551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.332759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.333267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.333722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.335087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.336783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.339290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 225/433] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:45.346557 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.432845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e3c47-902b-459b-abc7-7fe188fc2421 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.436056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.474260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.474971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.475296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.475581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.476401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] RBAC: Adding query filter params (name=member) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:45.476717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.479024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.483735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014286a3-b3d4-44dd-8ce8-ed8d30b9b5ae None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.489207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 209/434] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:45.495217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.528080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45242fa-6650-4c5f-bab7-8f367356ae52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.529559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45242fa-6650-4c5f-bab7-8f367356ae52 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.530060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45242fa-6650-4c5f-bab7-8f367356ae52 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.530897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45242fa-6650-4c5f-bab7-8f367356ae52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.532200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45242fa-6650-4c5f-bab7-8f367356ae52 None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=2edc2b4730164d7195bc4fc982443c5b) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:45.532693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45242fa-6650-4c5f-bab7-8f367356ae52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.535276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45242fa-6650-4c5f-bab7-8f367356ae52 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.543639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 226/435] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=2edc2b4730164d7195bc4fc982443c5b&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:45.602497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa993-8e7f-47ec-a1a4-945bdb1ea1d8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.602812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa993-8e7f-47ec-a1a4-945bdb1ea1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.603732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa993-8e7f-47ec-a1a4-945bdb1ea1d8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.605444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 227/436] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:45.611656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab5013-3d21-4ae9-95b6-13febd8bb86c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.612299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab5013-3d21-4ae9-95b6-13febd8bb86c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.612790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab5013-3d21-4ae9-95b6-13febd8bb86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.617752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca609-7ee9-4fe6-be53-af30425a425f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.633177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-feab5013-3d21-4ae9-95b6-13febd8bb86c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:45.634831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-feab5013-3d21-4ae9-95b6-13febd8bb86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:45.662090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.663370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.663663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.663944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.688139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.722336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:45.727836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 228/437] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:45.752307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:45.753827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:45.754755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab5013-3d21-4ae9-95b6-13febd8bb86c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.876291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 210/438] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:45.882022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.886672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab5013-3d21-4ae9-95b6-13febd8bb86c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:45.933510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:45.934350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:45.934656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:45.934977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:45.949333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:45.960210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:45.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:45.962026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:45.967341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 229/439] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:45.969375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:45.977557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.013047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.031343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6cb9b-903a-4c10-974c-31a2f63a2fcc None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.084722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.084722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.084722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.084722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.086426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.088624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.092143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 211/440] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.099721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.104436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21008de-e9b2-45ff-8212-db0c195d08a1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.141175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad150c70-10b2-4804-beae-263dd7f9032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.141849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad150c70-10b2-4804-beae-263dd7f9032e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.142139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad150c70-10b2-4804-beae-263dd7f9032e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.142425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad150c70-10b2-4804-beae-263dd7f9032e None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.143201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad150c70-10b2-4804-beae-263dd7f9032e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.143534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad150c70-10b2-4804-beae-263dd7f9032e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.144931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad150c70-10b2-4804-beae-263dd7f9032e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.152941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 230/441] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.154219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ff84b-df4c-4b92-8023-197c08189abc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.154560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ff84b-df4c-4b92-8023-197c08189abc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.154794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ff84b-df4c-4b92-8023-197c08189abc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.156205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 231/442] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:46.160168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ff84b-df4c-4b92-8023-197c08189abc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.208145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b31d5-6580-4f47-b27d-b12a9dcf4e21 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.241398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7527eb-fc51-474a-86bd-7ad30a98070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.242260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7527eb-fc51-474a-86bd-7ad30a98070a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.242549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7527eb-fc51-474a-86bd-7ad30a98070a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.242831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7527eb-fc51-474a-86bd-7ad30a98070a None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.244265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7527eb-fc51-474a-86bd-7ad30a98070a None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.245730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7527eb-fc51-474a-86bd-7ad30a98070a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.247375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 212/443] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.248600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbdbbb-52eb-4395-99c5-71516bb9cbcf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.248899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbdbbb-52eb-4395-99c5-71516bb9cbcf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.249184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbdbbb-52eb-4395-99c5-71516bb9cbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.251736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.254818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.262102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ff84b-df4c-4b92-8023-197c08189abc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.268093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-05cbdbbb-52eb-4395-99c5-71516bb9cbcf None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:46.269277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-05cbdbbb-52eb-4395-99c5-71516bb9cbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:46.296237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.296893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.297183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.297468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.334955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 213/444] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:46.336411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbdbbb-52eb-4395-99c5-71516bb9cbcf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.369113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.370577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.436080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbdbbb-52eb-4395-99c5-71516bb9cbcf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.457766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.465075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.467044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 232/445] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:46.467784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.474921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.475608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.475896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.476581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.477451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=adb9ae992e5a4594a8f7510556b4db5f, scope.project.id=3619df4b249349909961aab754e48694) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.477771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.480164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.486098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 214/446] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=3619df4b249349909961aab754e48694&role.id=adb9ae992e5a4594a8f7510556b4db5f => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.486965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.589791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf721bb-144a-4de1-919d-3ae10a4812f1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.602140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5b1ea-ed7a-4309-a2a4-63d4ada8af80 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.628748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.629555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.629836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.630116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.643024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.643932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.644052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:46.644230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.644491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] PATH_INFO: `/v3/projects/3619df4b249349909961aab754e48694` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.645692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] RBAC: Authorizing `identity:get_project(project_id=3619df4b249349909961aab754e48694)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.647140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.649575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 215/447] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/projects/3619df4b249349909961aab754e48694 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.654945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:46.657007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:46.657902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:46.658715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 233/448] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:46.663978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.729054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.731233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.771514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b40211-c5ca-43b9-8657-173d862abdc5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.789179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad7a57-6138-4e1e-ba2f-4dfdef11d89a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.816180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.816573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.816864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.817141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] PATH_INFO: `/v3/groups/c84c932d1f964502b210a1b802366260` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.819910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] RBAC: Authorizing `identity:get_group(group_id=c84c932d1f964502b210a1b802366260)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.821453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.823431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 216/449] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/groups/c84c932d1f964502b210a1b802366260 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:46.830580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.835213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.835895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.836178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.836455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.837226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.838477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.840219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.849375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 234/450] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.856216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:46.941386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f2d17-1757-4273-9a13-e708b9f0083b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.959873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9203e-1135-4b6f-8417-6313a76b59d5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:46.970870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.973755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:46.975697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.976354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.976640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.976924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:46.977752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] RBAC: Adding query filter params (group.id=c84c932d1f964502b210a1b802366260, role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=3619df4b249349909961aab754e48694) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:46.978607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:46.984980 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:46.985780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 217/451] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/role_assignments?group.id=c84c932d1f964502b210a1b802366260&scope.project.id=3619df4b249349909961aab754e48694&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:46.996038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:46.996854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:46.997268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:46.997559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.072973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.074455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.176465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 235/452] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 10:49:47.182749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.286863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd620f5-de9b-4743-a5e4-96d9458c5a2a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.320517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.321375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.321656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.321939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] PATH_INFO: `/v3/projects/d945ea14d2824c65a22104765fba4b15` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.323324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] RBAC: Authorizing `identity:get_project(project_id=d945ea14d2824c65a22104765fba4b15)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.324757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.326441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 218/453] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/projects/d945ea14d2824c65a22104765fba4b15 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:47.331905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.347495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.427494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac67da-fc73-44b9-9bf5-57101d351e69 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.459938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c44cc-1ee7-4c48-8bce-3138ac0c2e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.460786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c44cc-1ee7-4c48-8bce-3138ac0c2e81 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.461074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c44cc-1ee7-4c48-8bce-3138ac0c2e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.461357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c44cc-1ee7-4c48-8bce-3138ac0c2e81 None admin] PATH_INFO: `/v3/groups/91837d0847c44e65a535025a25ef1e16` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.463032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c44cc-1ee7-4c48-8bce-3138ac0c2e81 None admin] RBAC: Authorizing `identity:get_group(group_id=91837d0847c44e65a535025a25ef1e16)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.464561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c44cc-1ee7-4c48-8bce-3138ac0c2e81 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.466162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 236/454] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/groups/91837d0847c44e65a535025a25ef1e16 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:47.471792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:47.533154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:47.569039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ba75f-2f3b-42fd-b273-66d196c41d7a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:47.602443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f21692-3041-4c6d-8db8-be81cd713581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:47.603787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f21692-3041-4c6d-8db8-be81cd713581 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:47.604082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f21692-3041-4c6d-8db8-be81cd713581 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:47.604365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f21692-3041-4c6d-8db8-be81cd713581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:47.605197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f21692-3041-4c6d-8db8-be81cd713581 None admin] RBAC: Adding query filter params (group.id=91837d0847c44e65a535025a25ef1e16, role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=d945ea14d2824c65a22104765fba4b15) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:47.605513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f21692-3041-4c6d-8db8-be81cd713581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:47.606929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f21692-3041-4c6d-8db8-be81cd713581 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:47.612610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 219/455] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 10:49:47 2022] GET /identity/v3/role_assignments?group.id=91837d0847c44e65a535025a25ef1e16&scope.project.id=d945ea14d2824c65a22104765fba4b15&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:50.004439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056964ed-4c5d-4306-9484-53677667a8a6 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.004992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056964ed-4c5d-4306-9484-53677667a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.004992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056964ed-4c5d-4306-9484-53677667a8a6 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.006678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 237/456] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:50.012808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173ebfa-e755-4d9e-8978-7afb202da697 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.013723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173ebfa-e755-4d9e-8978-7afb202da697 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.014288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173ebfa-e755-4d9e-8978-7afb202da697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.040157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4173ebfa-e755-4d9e-8978-7afb202da697 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:50.041738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4173ebfa-e755-4d9e-8978-7afb202da697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:50.129965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 220/457] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:50.179087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929b63-573b-4a5a-870b-eef2c304018d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.179378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929b63-573b-4a5a-870b-eef2c304018d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.179604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929b63-573b-4a5a-870b-eef2c304018d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.181034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 238/458] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:50.182193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173ebfa-e755-4d9e-8978-7afb202da697 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.190212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977f54e-9c7d-435a-9c78-3c9d52d30d42 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.190508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977f54e-9c7d-435a-9c78-3c9d52d30d42 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.190790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977f54e-9c7d-435a-9c78-3c9d52d30d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.212022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2977f54e-9c7d-435a-9c78-3c9d52d30d42 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:50.213970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2977f54e-9c7d-435a-9c78-3c9d52d30d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:50.295823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.333058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 239/459] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:50.361555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173ebfa-e755-4d9e-8978-7afb202da697 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.388235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977f54e-9c7d-435a-9c78-3c9d52d30d42 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.422709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.435528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c51bf5-1496-4cb9-9250-de38ce1c0a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.436265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51bf5-1496-4cb9-9250-de38ce1c0a8f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.436545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51bf5-1496-4cb9-9250-de38ce1c0a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.436821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51bf5-1496-4cb9-9250-de38ce1c0a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.457048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c51bf5-1496-4cb9-9250-de38ce1c0a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.458593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c51bf5-1496-4cb9-9250-de38ce1c0a8f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-59c51bf5-1496-4cb9-9250-de38ce1c0a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:50.488679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:50.489612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:50.490469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:50.490469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:50.490469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:50.490469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:50.490469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 221/460] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:50.491243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3973a-7338-49d0-afb9-cc113f14bcc9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.491534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3973a-7338-49d0-afb9-cc113f14bcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.491815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3973a-7338-49d0-afb9-cc113f14bcc9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.494094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 222/461] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:50.500758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3524e-73b5-409f-8096-2cc4cc6103d2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.501057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3524e-73b5-409f-8096-2cc4cc6103d2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.501458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3524e-73b5-409f-8096-2cc4cc6103d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.520826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-80f3524e-73b5-409f-8096-2cc4cc6103d2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:50.522386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-80f3524e-73b5-409f-8096-2cc4cc6103d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:50.542718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977f54e-9c7d-435a-9c78-3c9d52d30d42 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.579306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.580118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.580409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.580684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.591852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.593533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.607327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 223/462] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:50.608285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3524e-73b5-409f-8096-2cc4cc6103d2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:50.613594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:50.614477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:50.615371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:50.615371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:50.615371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:50.615371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:50.617776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 240/463] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:50.624628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.673622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.724169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.748985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3524e-73b5-409f-8096-2cc4cc6103d2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.765838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840f90-030e-4b67-991d-1939c485036a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.794948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.796290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.796736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.797473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.798354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:50.798955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.801227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.804521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.805440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.806006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.807521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:50.808474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:50.808988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:50.810614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:50.815211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 224/464] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:50.816035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.829963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 241/465] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:50.830875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:50.947945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:50.951567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075239c-407b-46f7-b4ac-c23af4670b7d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.973818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13407e3a-362b-4252-8985-3cbadc3281e1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:50.991653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:50.992387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:50.992671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:50.992945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.003458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.004907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.006900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.019194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90834293-ec45-4c51-a72b-6041e93a4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.019879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834293-ec45-4c51-a72b-6041e93a4b6e None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.024008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834293-ec45-4c51-a72b-6041e93a4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.024008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834293-ec45-4c51-a72b-6041e93a4b6e None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.024008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90834293-ec45-4c51-a72b-6041e93a4b6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.024008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90834293-ec45-4c51-a72b-6041e93a4b6e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:51.024284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:51.030940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:51.031932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:51.031932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:51.031932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.031932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:51.031932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 225/466] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:51.031932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.130341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f819311-4813-41d4-9778-ac84380fbeb2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.142112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 242/467] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:50 2022] POST /identity/v3/users => generated 246 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:51.142112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473426-40e8-4760-9087-44291ea0cf5e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.142112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473426-40e8-4760-9087-44291ea0cf5e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.142112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67473426-40e8-4760-9087-44291ea0cf5e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.145103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 243/468] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:51.145103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473426-40e8-4760-9087-44291ea0cf5e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.175667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4459e2d-0b12-4b1c-9599-821babe8af77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.176648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4459e2d-0b12-4b1c-9599-821babe8af77 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.177087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4459e2d-0b12-4b1c-9599-821babe8af77 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.177518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4459e2d-0b12-4b1c-9599-821babe8af77 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.178463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4459e2d-0b12-4b1c-9599-821babe8af77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.180119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4459e2d-0b12-4b1c-9599-821babe8af77 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.261034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67473426-40e8-4760-9087-44291ea0cf5e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.281878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 226/469] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:51 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:51.281878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661559ae-791d-4fbb-98d3-ca0124d6a595 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.281878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661559ae-791d-4fbb-98d3-ca0124d6a595 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.281878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661559ae-791d-4fbb-98d3-ca0124d6a595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.314179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-661559ae-791d-4fbb-98d3-ca0124d6a595 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:51.315461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-661559ae-791d-4fbb-98d3-ca0124d6a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:51.317974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.325801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.333536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.333536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.333536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.333536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:51.333536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.333536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.349040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 244/470] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:51.354818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.411627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 227/471] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:51.415631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.445114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661559ae-791d-4fbb-98d3-ca0124d6a595 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.489248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888c92f-efe0-4282-96ea-718ce19727d9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.538895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.539946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.540994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.541471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.542345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.544002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.573596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661559ae-791d-4fbb-98d3-ca0124d6a595 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.613551 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214da9-59cc-4b64-b87a-984870f00b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.614365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214da9-59cc-4b64-b87a-984870f00b39 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.614652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214da9-59cc-4b64-b87a-984870f00b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.614933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214da9-59cc-4b64-b87a-984870f00b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.622697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.626596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c214da9-59cc-4b64-b87a-984870f00b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.628100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c214da9-59cc-4b64-b87a-984870f00b39 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9c214da9-59cc-4b64-b87a-984870f00b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:51.649545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:51.653772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 228/472] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5781c-fc2a-4366-b719-727f1bc2e17d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5781c-fc2a-4366-b719-727f1bc2e17d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.654653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5781c-fc2a-4366-b719-727f1bc2e17d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.655506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 229/473] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:51.660599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653789b0-643f-473a-a9d5-23ba0f1906c8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.660889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653789b0-643f-473a-a9d5-23ba0f1906c8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.661175 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653789b0-643f-473a-a9d5-23ba0f1906c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.678400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-653789b0-643f-473a-a9d5-23ba0f1906c8 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:51.679647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-653789b0-643f-473a-a9d5-23ba0f1906c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:51.682095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 245/474] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:51 2022] POST /identity/v3/users => generated 248 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:51.687729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.790264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 230/475] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:51.819328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:51.820454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653789b0-643f-473a-a9d5-23ba0f1906c8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.833894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a632c0a-cba6-40a8-9265-362e68110bb2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:51.906719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:51.907440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:51.907738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:51.908027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:51.909146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:51.909546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:51.911032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:51.927778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 246/476] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:51.933425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:51.963974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653789b0-643f-473a-a9d5-23ba0f1906c8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.018441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.019126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.019446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.019726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.020483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.021867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.027960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.069786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002ceaf-bb1c-4c76-9447-1b5f2bbee857 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.074021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.sql.core [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 10:49:52.074021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 10:49:52.074021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [parameters: {'id': '0d8a1e2e37dd4164b3a0675eb5053b72', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 10:49:52.074021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80342) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result.read() Jun 23 10:49:52.099012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:49:52.100028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application yield resource Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise self.value Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application session.commit() Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 10:49:52.100939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.session.flush() Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application compat.raise_( Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 10:49:52.101809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:49:52.104133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application result.read() Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:49:52.106160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [parameters: {'id': '0d8a1e2e37dd4164b3a0675eb5053b72', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:52.108180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 10:49:52.110337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:52.112640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 231/477] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 10:49:51 2022] POST /identity/v3/roles => generated 144 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:52.112640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.112936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.112936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.112936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.112936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.113521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.116624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.205250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 247/478] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:51 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:52.246847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68783049-d53c-49a3-87b4-7bcad48641ba None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.285948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6fa35-a2e2-47be-91fc-bb6d9c0f9982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.286762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6fa35-a2e2-47be-91fc-bb6d9c0f9982 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.287045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6fa35-a2e2-47be-91fc-bb6d9c0f9982 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.287355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6fa35-a2e2-47be-91fc-bb6d9c0f9982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.300464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6fa35-a2e2-47be-91fc-bb6d9c0f9982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-edd6fa35-a2e2-47be-91fc-bb6d9c0f9982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:52.308836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:52.309716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:52.310920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 232/479] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:52.316159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:52.333782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:52.427420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb418887-f247-4fb1-9de1-6f0584fd1833 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:52.465723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024bffe-24e6-4fb5-8f08-cdace2fc2127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:52.466410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024bffe-24e6-4fb5-8f08-cdace2fc2127 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:52.466695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024bffe-24e6-4fb5-8f08-cdace2fc2127 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:52.466971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024bffe-24e6-4fb5-8f08-cdace2fc2127 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:52.467772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024bffe-24e6-4fb5-8f08-cdace2fc2127 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:52.468087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024bffe-24e6-4fb5-8f08-cdace2fc2127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:52.469576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024bffe-24e6-4fb5-8f08-cdace2fc2127 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:52.479394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 248/480] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:53.548580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73332962-47b8-4d64-b5c2-a3c848d1d5f0 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.549099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73332962-47b8-4d64-b5c2-a3c848d1d5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.549176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73332962-47b8-4d64-b5c2-a3c848d1d5f0 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.550727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 233/481] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:53.556692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2a3b4-5153-4a6e-88d0-9e07a803dd84 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.557486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2a3b4-5153-4a6e-88d0-9e07a803dd84 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.557963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2a3b4-5153-4a6e-88d0-9e07a803dd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.584171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9cc2a3b4-5153-4a6e-88d0-9e07a803dd84 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:53.585709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9cc2a3b4-5153-4a6e-88d0-9e07a803dd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:53.678769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 249/482] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:53.713833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73332962-47b8-4d64-b5c2-a3c848d1d5f0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.774335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:53.852505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73332962-47b8-4d64-b5c2-a3c848d1d5f0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:53.901305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb49b8f-d034-45b0-9cb1-cc2850933545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:53.902104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb49b8f-d034-45b0-9cb1-cc2850933545 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:53.902389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb49b8f-d034-45b0-9cb1-cc2850933545 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:53.902667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb49b8f-d034-45b0-9cb1-cc2850933545 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:53.923110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb49b8f-d034-45b0-9cb1-cc2850933545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-2cb49b8f-d034-45b0-9cb1-cc2850933545 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:53.931201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:53.932237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:53.933154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:53.936906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 234/483] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:53.938238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2a3b4-5153-4a6e-88d0-9e07a803dd84 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:53.941372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.071191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2a3b4-5153-4a6e-88d0-9e07a803dd84 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.094731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba63b6-4eb6-47c7-88e7-664341113285 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.095037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba63b6-4eb6-47c7-88e7-664341113285 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.095321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba63b6-4eb6-47c7-88e7-664341113285 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.096690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 235/484] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:54.102401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31ddc5-8639-455f-9cb9-a8e4a095855e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.102945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31ddc5-8639-455f-9cb9-a8e4a095855e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.103118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31ddc5-8639-455f-9cb9-a8e4a095855e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.126587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.127276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.127563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.127844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.128595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.128903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.130271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.142058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ab31ddc5-8639-455f-9cb9-a8e4a095855e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:54.143300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ab31ddc5-8639-455f-9cb9-a8e4a095855e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:54.147467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 250/485] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.153603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.222704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.259641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122259c8-909c-4277-ba64-af7054fef131 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.308176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.308986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.309275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.309555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.329908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 236/486] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:54.331999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.333473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.342075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80917e-a5b0-489a-8f55-2df70c11dd24 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.342665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80917e-a5b0-489a-8f55-2df70c11dd24 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.343258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80917e-a5b0-489a-8f55-2df70c11dd24 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.345058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 237/487] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:49:54.350320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610c10b-8528-4bed-a36a-fd68f0558b2f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.350818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610c10b-8528-4bed-a36a-fd68f0558b2f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.351795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610c10b-8528-4bed-a36a-fd68f0558b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:54.352342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:54.353780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:54.354671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:54.354671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:54.354671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:54.354671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:54.354671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 251/488] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:54.360163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.375488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d610c10b-8528-4bed-a36a-fd68f0558b2f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:54.377089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d610c10b-8528-4bed-a36a-fd68f0558b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:54.431585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.463204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 238/489] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:49:54.463204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610c10b-8528-4bed-a36a-fd68f0558b2f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.489045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811c74a-6570-4823-a2b4-b80f41fa5afa None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.517041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.536288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.537105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.537494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.537871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.538706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.539115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.540821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.553830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 252/490] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.558637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.574506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610c10b-8528-4bed-a36a-fd68f0558b2f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.615586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.616551 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.616920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.617294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.629043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:54.638193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:54.639301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:54.640329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:54.642466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 239/491] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:54.643481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.677338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c31e75-eaf3-4380-b105-98838663fae7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.687555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.716922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.717870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.718257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.718635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.730762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:54.738786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:54.739857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:54.740865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:54.747472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 253/492] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:54.748705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.772611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0ef5d-0faf-4c0b-8315-4a620717b740 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.799865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:54.812663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.813590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.813947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.818245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.823077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.824700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:54.841770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:54.842977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:49:54.844089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:54.850543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 240/493] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:54.850543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.892298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720264d-041d-41c9-90c6-8356f84cd59d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:54.972504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:54.973889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:54.974178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:54.974467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:54.975249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:54.975571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:54.976960 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:54.985704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 254/494] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:54.986539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:54.993871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.012294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94ce2-e19f-4989-9fd5-c8d5bbc6a75b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.048977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa3f46-9c1e-4567-8801-1b47a04929d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.049637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3f46-9c1e-4567-8801-1b47a04929d6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.049933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3f46-9c1e-4567-8801-1b47a04929d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.050216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3f46-9c1e-4567-8801-1b47a04929d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.050972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa3f46-9c1e-4567-8801-1b47a04929d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.051315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa3f46-9c1e-4567-8801-1b47a04929d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.052716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa3f46-9c1e-4567-8801-1b47a04929d6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.061580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 241/495] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.063920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae42f5-5b9e-4d31-b64f-efd48fb2c692 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.064220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae42f5-5b9e-4d31-b64f-efd48fb2c692 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.064504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae42f5-5b9e-4d31-b64f-efd48fb2c692 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.065759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 242/496] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:55.066721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae42f5-5b9e-4d31-b64f-efd48fb2c692 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.089962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.092901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6f479-d754-4bcb-ba73-4af6a9223494 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.125985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf2ed4-b05b-4709-b512-4e5114049095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.126662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf2ed4-b05b-4709-b512-4e5114049095 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.126942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf2ed4-b05b-4709-b512-4e5114049095 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.127248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf2ed4-b05b-4709-b512-4e5114049095 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.128135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf2ed4-b05b-4709-b512-4e5114049095 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.128408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf2ed4-b05b-4709-b512-4e5114049095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.130466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf2ed4-b05b-4709-b512-4e5114049095 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.168485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 255/497] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:49:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.169702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc1f3f-d2ac-4ffb-86b0-66b1a0f44bf7 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.169996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc1f3f-d2ac-4ffb-86b0-66b1a0f44bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.170277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc1f3f-d2ac-4ffb-86b0-66b1a0f44bf7 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.170764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae42f5-5b9e-4d31-b64f-efd48fb2c692 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.171541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 256/498] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:55.172242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc1f3f-d2ac-4ffb-86b0-66b1a0f44bf7 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.207497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc9432-982c-4ae9-923b-f6dfc0522772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.208287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9432-982c-4ae9-923b-f6dfc0522772 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.208571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9432-982c-4ae9-923b-f6dfc0522772 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.208849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc9432-982c-4ae9-923b-f6dfc0522772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.220015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9432-982c-4ae9-923b-f6dfc0522772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.221482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc9432-982c-4ae9-923b-f6dfc0522772 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.229013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c5bc9432-982c-4ae9-923b-f6dfc0522772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:55.240136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.242678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 243/499] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb63d-1cf2-4e3a-9e03-17e120f9d57e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb63d-1cf2-4e3a-9e03-17e120f9d57e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.243596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb63d-1cf2-4e3a-9e03-17e120f9d57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.260284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1cccb63d-1cf2-4e3a-9e03-17e120f9d57e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:55.262090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1cccb63d-1cf2-4e3a-9e03-17e120f9d57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:55.283205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc1f3f-d2ac-4ffb-86b0-66b1a0f44bf7 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.317327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379e29c-0359-4b7c-aee3-f9355307e8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.318110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379e29c-0359-4b7c-aee3-f9355307e8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.318391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379e29c-0359-4b7c-aee3-f9355307e8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.318664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379e29c-0359-4b7c-aee3-f9355307e8c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.329113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379e29c-0359-4b7c-aee3-f9355307e8c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.330103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 244/500] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:55.330581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379e29c-0359-4b7c-aee3-f9355307e8c6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.330969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb63d-1cf2-4e3a-9e03-17e120f9d57e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8379e29c-0359-4b7c-aee3-f9355307e8c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:55.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.358262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.359134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:55.359134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:55.359134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.359134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:55.359134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 257/501] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.359798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f662f9f-20c8-4e0a-a70f-ae3248ffe1ca None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.364175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f662f9f-20c8-4e0a-a70f-ae3248ffe1ca None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.364175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f662f9f-20c8-4e0a-a70f-ae3248ffe1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.376813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0f662f9f-20c8-4e0a-a70f-ae3248ffe1ca None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:55.377999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0f662f9f-20c8-4e0a-a70f-ae3248ffe1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:55.381071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.431401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb63d-1cf2-4e3a-9e03-17e120f9d57e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.443819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 258/502] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:55.444645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f662f9f-20c8-4e0a-a70f-ae3248ffe1ca None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.457949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.466848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.467541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.467830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.468114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.536100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.537557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.544760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f662f9f-20c8-4e0a-a70f-ae3248ffe1ca None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.577243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.577904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.578191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.578470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.579293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.579613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.581002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.590335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.593604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 259/503] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.594391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.635349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 245/504] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:55.636184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.668506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.698128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436aff8-1a7c-45b2-8ff2-37645682e313 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.732562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.733363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.733655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.733933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.739703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb536c6-9ad6-416c-9b19-08161afaaedc None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.745239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.746709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:55.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.764628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.765472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:55.765472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:55.765472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:55.765472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:55.765472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 260/505] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.765760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.779339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.780008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.780296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.780574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.781620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:55.781620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.783014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.794184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 246/506] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:55.794964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.817725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.867070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba944605-97fe-4c20-bcbc-f5554cee625a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.884712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:55.897034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead1426-96f2-4e4b-b1a3-f2753e8022c2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:55.900853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.901650 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.901935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.902214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.912676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:55.921212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.927112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 261/507] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.927976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:55.935441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:55.936245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:55.936538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:55.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:55.943909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:55.945359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:55.959927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:55.961001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 247/508] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:55.962798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.029110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad0778-2768-4131-9758-3ecad8d315b1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.041479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.061414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.062212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.062499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.062780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.063897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffd403-5ee6-48d2-b6b3-261548c36899 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.074370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.075916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:56.093171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.094172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.095052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:56.095052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:56.095052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.095052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:56.095052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 262/509] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.095358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.102405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.103237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.103527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.103810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.104533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.104849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.106236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.118342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 248/510] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.119130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.129512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.201486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eb4a6-b61b-4d57-a2a6-26a8daa740cf None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.219242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e01293-bdcf-44a7-af60-a7e10b6884a8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.236226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.237027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.237309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.237581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.244875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.246313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.253135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.253885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.254244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.254596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.255451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.255839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.257269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:56.262696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.263773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:56.264747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 263/511] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.265302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.269510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 249/512] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.270374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.290962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.339436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.365090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dde66-546a-4aa6-8dce-6654586247de None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.370404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e472b07-e33e-4ace-a2df-3d0f52c69e01 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.398935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.399825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.400125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.400405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.401236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.401551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.402982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.403667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.404427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.404719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.404999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.405719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.406028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.407389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.419112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 250/513] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.424506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 264/514] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.424640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.424756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.502966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.520717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c97b-75e8-4906-8d49-8f1957f477a9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.520931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca54aa-7d8a-4b51-851b-ca0536d95eda None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.549371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.554662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.555379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.556199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.556480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.556720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.557188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.557408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.557695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.557975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.558626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.558813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:56.559136 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.560627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.578116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 265/515] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:56.578917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.672596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 251/516] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:49:56 2022] POST /identity/v3/users => generated 247 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:49:56.673639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.686755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fb23b-3656-4db4-ad37-4af45c9deb54 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.726756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.728474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.728798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.729137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.739081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.742223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.744529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:56.766016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.767833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:56.767833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:56.767833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:56.767833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:56.768571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 266/517] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.769057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.802352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de6fb4-decf-4f44-939a-8b3d2a7ed587 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.846837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.847674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.848217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.848498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.857606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:56.859083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:56.877524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:56.878441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:56.879323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 252/518] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:56.879793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:56.892551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d2a66-a177-4ff6-825e-5b5e24b30da7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:56.930879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:56.931657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:56.931949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:56.932234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:56.944106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:56.995656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.005695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.007285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.008463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2189d4-ea25-4d80-8cc4-198afcda3546 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.045728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.046306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.046595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.046877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.115486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 267/519] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:57.116178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.121350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.122839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.167380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.220962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.226692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 253/520] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:57.226692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.235945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c332d-6f96-48f0-a5a9-1b3a57c4943d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.272183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.272857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.273147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.273431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.274902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.275335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.277436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.289367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 268/521] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.290620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.342572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce174-b6d9-43e4-99c0-383b8aa7f2b9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.382416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.383091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.383412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.384519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.384519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.384784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.386224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.396541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759fed3-904e-479b-8b35-8566c5b944d4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.405133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 254/522] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.405781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.414647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.438996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.439811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.440097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.440375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.452072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.453538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:57.472190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:57.473398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:57.479461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:57.479461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:57.479461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:57.479461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:57.479461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 269/523] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:57.479461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.479461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.520780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ed300-173c-41e6-a1fa-a04f26f74c3c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.557233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.558136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.558431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.558707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.569828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.571965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.578646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e69980-710e-4d38-b5e8-40b6326e709e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:57.594102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:57.595050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:57.596287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:57.596287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:57.596287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:57.596287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:57.596287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 255/524] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:57.597746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.619089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.620254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.620737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.621318 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.631166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.633329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:49:57.648221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:57.649337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 270/525] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:57.650592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.671948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.720777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.723980 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f927d-7dd8-4002-a160-17c68ee69346 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.760070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.760843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.761136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.761414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.762251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=54bb42878fff477099bbf56a46e9eec0) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.762348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233d5f6-695f-49dd-a596-e19294a951c8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.762595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.764040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.779210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 256/526] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/role_assignments?user.id=54bb42878fff477099bbf56a46e9eec0&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.779210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.824802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.825938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.826225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.827065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.828146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.828464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.830204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.850591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 271/527] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.853593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.892370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f51d9e-bbbe-47a0-a56b-b69ba63d0551 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:57.931122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:57.946023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:57.947521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:57.948161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:57.948512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:57.949453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:57.949763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:57.951671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:57.970569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 257/528] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:57.972414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:57.997719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d3ffc-194e-4e2f-ac9e-78d0bbbccefb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.022569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.034532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.035470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.035918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.036361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.037340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:58.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.039420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.060669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 272/529] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:58.061867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.107365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cbd2b-877b-415f-a6b3-4e04c7498116 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.156559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.157511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.157908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.158301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.169515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.170959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.178912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffd4b7-0bdf-43c6-b6c5-3c93cfc5df12 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:58.185161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:58.186001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:58.186837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 258/530] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:58.187759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.228238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.229411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.229893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.230364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.235745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.241384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.243749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:58.266645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:58.268508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:58.269381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:58.273227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 273/531] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:58.275909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.309831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cb87e-63cf-44a9-9601-d33a5d0b8cbe None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.331062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.356013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.357505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.358216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.358673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.396649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64fb4c-15ce-4327-92ac-c7f9acec2221 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.436159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.436773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.437068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.437353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.438156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:58.438474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.440018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.454714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.457330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.459242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 274/532] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:58.460160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.517619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.576215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32992b-7920-4c80-abda-014a0c266ec7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.578479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.599863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 259/533] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:49:58.600832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.624322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.626204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.626973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.627477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.628755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:58.629648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.631850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.648890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 275/534] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:58.650027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.732777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162a801-fb42-4748-a0b5-c1cb9cb0fe65 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.770397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.771462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.771938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.772409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.773516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=282c8662f2a2431b97d1bf122eabf5a7) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:58.774120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.776558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.777640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930824ad-06c6-4441-8f8a-558bf03bd6dd None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.783547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 260/535] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/role_assignments?user.id=282c8662f2a2431b97d1bf122eabf5a7&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:58.784588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.794982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.844723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3423325-e485-47b7-afed-d9f26e4c1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.845841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3423325-e485-47b7-afed-d9f26e4c1084 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.846189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3423325-e485-47b7-afed-d9f26e4c1084 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.846479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3423325-e485-47b7-afed-d9f26e4c1084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.876994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3423325-e485-47b7-afed-d9f26e4c1084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.880509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3423325-e485-47b7-afed-d9f26e4c1084 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d3423325-e485-47b7-afed-d9f26e4c1084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:58.912555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:58.913536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:58.914467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:58.914467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:58.914467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:58.914467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:58.917530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 276/536] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:58.919291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38967-1a05-4740-a8d1-a95ec35a8a12 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.919802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38967-1a05-4740-a8d1-a95ec35a8a12 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.920287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38967-1a05-4740-a8d1-a95ec35a8a12 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.926480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 277/537] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:58.927777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a38967-1a05-4740-a8d1-a95ec35a8a12 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:58.936828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee790bb-5e83-4e52-b4a8-c5b51cb6cb12 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:58.947064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:58.982046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc75e0d-2772-4450-a000-2f3b27a2ab4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:58.982763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75e0d-2772-4450-a000-2f3b27a2ab4b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:58.983060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75e0d-2772-4450-a000-2f3b27a2ab4b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:58.983396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75e0d-2772-4450-a000-2f3b27a2ab4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:58.984859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75e0d-2772-4450-a000-2f3b27a2ab4b None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=2678deaa97fc4012aa0d71f9e505f5ca) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:58.985183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75e0d-2772-4450-a000-2f3b27a2ab4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:58.987013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75e0d-2772-4450-a000-2f3b27a2ab4b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:58.999264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 261/538] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/role_assignments?user.id=2678deaa97fc4012aa0d71f9e505f5ca&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:59.002820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26383a0-5926-418b-a4f8-62868c6f66b5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.005922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26383a0-5926-418b-a4f8-62868c6f66b5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.009957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26383a0-5926-418b-a4f8-62868c6f66b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.052238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a26383a0-5926-418b-a4f8-62868c6f66b5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:49:59.058044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a26383a0-5926-418b-a4f8-62868c6f66b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:49:59.126959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a38967-1a05-4740-a8d1-a95ec35a8a12 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.155227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.169139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 262/539] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:49:59.189739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.190415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.190703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.190982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.191751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.192061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.193565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.208450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 278/540] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:49:59.211126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26383a0-5926-418b-a4f8-62868c6f66b5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.215415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.339212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.360686 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26383a0-5926-418b-a4f8-62868c6f66b5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.365341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf14c-c70a-4d37-a6e6-df78a9852570 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.398343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.399617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.400123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.400620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.411889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:49:59.419117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.419989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.420274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.420607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:49:59.421227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:59.422112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:49:59.422950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:59.425668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 263/541] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:49:59.432202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.435330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.436811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:49:59.451670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:59.452561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:49:59.453422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 279/542] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:59.458773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.579565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.591214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39a217-7109-40e9-b8f8-cb58e0fd2c86 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.613346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718c38-c44b-47b3-9aa0-8fa6bb529273 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.656451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.657130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.657422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.657706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.658466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.658777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.660195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.681235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 264/543] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:59.682336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.683015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.683331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.683683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.684514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.684838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.686293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.688082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.689270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.710077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 280/544] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:59.716239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:49:59.869251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b9305-40f6-4891-b76f-aea6fefc2873 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.882069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e822a-cffa-43f5-964e-2bd86433358a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:49:59.940131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:49:59.958366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.960385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.960385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.960385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.965298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699ceb6-dec7-4643-b320-f0e305c2280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:49:59.966231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699ceb6-dec7-4643-b320-f0e305c2280a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.966743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699ceb6-dec7-4643-b320-f0e305c2280a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.967238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699ceb6-dec7-4643-b320-f0e305c2280a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.968727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699ceb6-dec7-4643-b320-f0e305c2280a None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=cf08ecfc3fed44e2bc1ccff8521a4ed8) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:49:59.969225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699ceb6-dec7-4643-b320-f0e305c2280a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.971562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:49:59.971739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699ceb6-dec7-4643-b320-f0e305c2280a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.973390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:49:59.980626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 281/545] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/role_assignments?user.id=cf08ecfc3fed44e2bc1ccff8521a4ed8&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:49:59.983204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e43fd3-3ae6-4310-894c-eb552230f227 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.984304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e43fd3-3ae6-4310-894c-eb552230f227 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.984760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e43fd3-3ae6-4310-894c-eb552230f227 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:49:59.986394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 282/546] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:49:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:49:59.992259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20ceb-c54f-4e4e-80e7-61443c568fd7 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:49:59.992804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20ceb-c54f-4e4e-80e7-61443c568fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:49:59.992963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:49:59.993882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:49:59.994722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:49:59.994722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:49:59.994722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:49:59.994722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:49:59.994722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 265/547] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:49:59.996137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20ceb-c54f-4e4e-80e7-61443c568fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.000222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ffe20ceb-c54f-4e4e-80e7-61443c568fd7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:00.017582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ffe20ceb-c54f-4e4e-80e7-61443c568fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:00.080703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.098001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 283/548] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:50:00.110561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bb8af-4fdf-4aa4-8dda-99468844ee1b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.123587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe20ceb-c54f-4e4e-80e7-61443c568fd7 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.147662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.148672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.148672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.148947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.150511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:00.150825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.152301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.165929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 266/549] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:00.171318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.207803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.243695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe20ceb-c54f-4e4e-80e7-61443c568fd7 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.278734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ef626-0bf7-4fe3-a38a-b9bcf7dfea83 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.282112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.283149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.283771 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.284276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.296098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:00.304474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:00.305896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:00.306727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:00.309689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 284/550] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:00.316394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.317059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.319274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.319557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.319891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.329344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.330468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:00.347300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:00.348583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:00.349515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 267/551] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:00.354695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.391703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.434946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bf0ef-40f8-4ce7-98a4-5bd20504f6f7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.463366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.469829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ed832-0990-4303-a243-6bd6e63ca702 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.480103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.481092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.481576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.482049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.487411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:00.487981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.489698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.506630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.506969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 285/552] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:00.511222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.511222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.511222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.511222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:00.511222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.511609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.514319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.558088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 268/553] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:00.563981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.679269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7c903-2ce4-4803-a335-42c8b5b86839 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.719780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.720394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.720685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.723226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.773337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.783827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a1f8-3f50-4ef2-a24a-d87f4847f8a3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:00.810408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:00.856109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:00.856903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:00.857249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:00.857525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:00.866846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.868916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.889019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:00.890519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:00.923199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:00.924108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:00.925082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:00.925082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:00.925082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:00.925082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:00.925082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 286/554] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:00.929616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:00.990034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 269/555] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:00.996647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.069440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.125971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088fc77-0f05-4ead-aa1d-ffd9da52d3f3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.162900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.163834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.164194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.164544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.176086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.177912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:01.197336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:01.199382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:01.201882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:01.201882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:01.201882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:01.201882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:01.201882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 270/556] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:01.207505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.223301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca534c1-9977-43c5-a0e9-f7b707c6283b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.289381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77bfc-035d-4cca-90b1-b1716313dd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.291063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77bfc-035d-4cca-90b1-b1716313dd6a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.291405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77bfc-035d-4cca-90b1-b1716313dd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.291690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77bfc-035d-4cca-90b1-b1716313dd6a None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.292435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77bfc-035d-4cca-90b1-b1716313dd6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:01.292747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77bfc-035d-4cca-90b1-b1716313dd6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.294133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77bfc-035d-4cca-90b1-b1716313dd6a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.307558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 287/557] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:01.308879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425fd2f-ef84-472d-b7be-0c7199ee1012 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.309171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425fd2f-ef84-472d-b7be-0c7199ee1012 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.309451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425fd2f-ef84-472d-b7be-0c7199ee1012 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.314902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 288/558] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:01.315696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425fd2f-ef84-472d-b7be-0c7199ee1012 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.331758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4d3de-1122-4b7d-947e-0fc74bf86aaf None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.335555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.343565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.379904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd210a-65c2-40e3-8b39-0d7100078946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.381256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd210a-65c2-40e3-8b39-0d7100078946 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.381938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd210a-65c2-40e3-8b39-0d7100078946 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.382386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd210a-65c2-40e3-8b39-0d7100078946 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.384078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd210a-65c2-40e3-8b39-0d7100078946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:01.385081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd210a-65c2-40e3-8b39-0d7100078946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.386902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd210a-65c2-40e3-8b39-0d7100078946 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.403716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 271/559] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:01.405590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb7337-0d5a-46c9-bcfb-b2118a4c1f6c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.406497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb7337-0d5a-46c9-bcfb-b2118a4c1f6c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.406953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb7337-0d5a-46c9-bcfb-b2118a4c1f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.428880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425fd2f-ef84-472d-b7be-0c7199ee1012 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.429602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ecbb7337-0d5a-46c9-bcfb-b2118a4c1f6c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:01.432053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ecbb7337-0d5a-46c9-bcfb-b2118a4c1f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:01.470565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.471498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.471840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.472161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.480204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.481772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:01.497310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:01.498495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:01.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:01.503630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 289/560] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:01.504502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.532746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 272/561] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:01.534037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb7337-0d5a-46c9-bcfb-b2118a4c1f6c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.614468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.633442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635df268-c435-4b55-b6e3-4b4c689cf55f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.646427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.653159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb7337-0d5a-46c9-bcfb-b2118a4c1f6c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.671344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.672448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.673338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.673689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.682931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.684963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.687671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20677d6-2f7a-42a4-997e-37b18bd25405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.688396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20677d6-2f7a-42a4-997e-37b18bd25405 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.688682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20677d6-2f7a-42a4-997e-37b18bd25405 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.688955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20677d6-2f7a-42a4-997e-37b18bd25405 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.689788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20677d6-2f7a-42a4-997e-37b18bd25405 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:01.690105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20677d6-2f7a-42a4-997e-37b18bd25405 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.691614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20677d6-2f7a-42a4-997e-37b18bd25405 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:01.701141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:01.702028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:01.703015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:01.710102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 290/562] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:01.710102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.710272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 273/563] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:01.712267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0ebe2-9cc0-4d3b-8cba-38c8ca503281 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.712688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0ebe2-9cc0-4d3b-8cba-38c8ca503281 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.713053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0ebe2-9cc0-4d3b-8cba-38c8ca503281 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.714552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 274/564] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:01.715286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0ebe2-9cc0-4d3b-8cba-38c8ca503281 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.813529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77016c-4804-43f9-903e-159089f3f042 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.821643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0ebe2-9cc0-4d3b-8cba-38c8ca503281 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:01.849858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.850794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.851197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.851598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.858827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710954e-0214-4a25-bad2-8e44cb74c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:01.859515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710954e-0214-4a25-bad2-8e44cb74c3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.861799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710954e-0214-4a25-bad2-8e44cb74c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.861799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710954e-0214-4a25-bad2-8e44cb74c3c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.861799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710954e-0214-4a25-bad2-8e44cb74c3c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:01.861799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710954e-0214-4a25-bad2-8e44cb74c3c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:01.862553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710954e-0214-4a25-bad2-8e44cb74c3c9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:01.862807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:01.872734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:01.873843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 291/565] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:01.876030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:01.878395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:01.883209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 275/566] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:01.883209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209fdef-2051-4a3a-8cfa-0be5f061811c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:01.883209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209fdef-2051-4a3a-8cfa-0be5f061811c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:01.883209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209fdef-2051-4a3a-8cfa-0be5f061811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:01.934863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9209fdef-2051-4a3a-8cfa-0be5f061811c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:01.936089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9209fdef-2051-4a3a-8cfa-0be5f061811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:01.944650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.014333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 276/567] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:02.015066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209fdef-2051-4a3a-8cfa-0be5f061811c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.032207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830d856-f06d-4d64-8016-b98036eebfe4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.067700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.068765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.069205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.069487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.081657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.083118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:02.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:02.105412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:02.106317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:02.106317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:02.106317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:02.106317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:02.106317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 292/568] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:02.106612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.129487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209fdef-2051-4a3a-8cfa-0be5f061811c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.168965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.171138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.171854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.172144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.172512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.173388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:02.173505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.174824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.188215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 277/569] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:02.188215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.189422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.221899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003754d-4c6b-49b3-a118-a6311f2c1b81 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.259137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.260191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.260710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.261292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.262455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=f766fee8481a4eccbc56d4af0c2afbf2) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:02.262883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.264353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.275269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 293/570] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/role_assignments?user.id=f766fee8481a4eccbc56d4af0c2afbf2&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:02.275269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.302398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e6089-e1e9-4bf6-b784-9564708e9078 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.342879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.343821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.344111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.344394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.359058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:02.367901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:02.368919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:02.369764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 278/571] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:02.375135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.417495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae49c4-0d7d-4229-8430-48ab845c9068 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.437594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.447534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.456436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.457791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.457791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.457791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.458375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:02.458687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.460104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.472887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 294/572] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:02.473721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.495815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f63789-558b-4147-af57-dfe4f5b80e26 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.535408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10650e69-c150-492b-8253-0f6e797e97b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.536223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10650e69-c150-492b-8253-0f6e797e97b7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.536508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10650e69-c150-492b-8253-0f6e797e97b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.536794 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10650e69-c150-492b-8253-0f6e797e97b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.548420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10650e69-c150-492b-8253-0f6e797e97b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.549940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10650e69-c150-492b-8253-0f6e797e97b7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-10650e69-c150-492b-8253-0f6e797e97b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:02.567711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:02.568628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:02.569556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:02.569556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:02.569556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:02.569556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:02.569556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 279/573] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:02.570375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5cc56-3917-4b3b-9b40-e63f837a79cf None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.570375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5cc56-3917-4b3b-9b40-e63f837a79cf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.570674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a5cc56-3917-4b3b-9b40-e63f837a79cf None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.572308 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 280/574] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:02.572850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5cc56-3917-4b3b-9b40-e63f837a79cf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.588426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6905f5-347e-4aaa-8146-3a601c457824 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.624966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.625658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.625941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.626215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.626964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:02.627298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.628654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.637481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 295/575] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:02.638211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.680668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.686534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a5cc56-3917-4b3b-9b40-e63f837a79cf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.708412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.728574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995558c-5d3f-455c-af23-808409cd2d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.729382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995558c-5d3f-455c-af23-808409cd2d0a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.729669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995558c-5d3f-455c-af23-808409cd2d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.729949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995558c-5d3f-455c-af23-808409cd2d0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.737246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995558c-5d3f-455c-af23-808409cd2d0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.739205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995558c-5d3f-455c-af23-808409cd2d0a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.741300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45327d57-1ec0-4997-9d19-cf14a950e1c2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0995558c-5d3f-455c-af23-808409cd2d0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:02.753737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:02.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:02.755820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 281/576] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:02.756335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ef10b-7c30-4db6-b469-bc4b0a397402 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.756883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ef10b-7c30-4db6-b469-bc4b0a397402 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.756883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ef10b-7c30-4db6-b469-bc4b0a397402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.773701 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7c6ef10b-7c30-4db6-b469-bc4b0a397402 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:02.775292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7c6ef10b-7c30-4db6-b469-bc4b0a397402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:02.784676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.785430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.785717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.786113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.786848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:02.787161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.788551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:02.806697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 296/577] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:02.806697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.859203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 282/578] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:02.859203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ef10b-7c30-4db6-b469-bc4b0a397402 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:02.939282 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.943489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d88903-2aeb-4eee-97b9-365a22914743 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.978034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:02.978763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:02.979045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:02.979353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:02.979453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ef10b-7c30-4db6-b469-bc4b0a397402 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:02.987150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:02.990323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:02.991853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:03.010568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.011575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.012539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:03.012539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:03.012539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.012539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:03.012539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 297/579] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.012539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.024692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.025769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.025769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.025934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.026685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:03.026997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.028451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.040644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 283/580] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:03.041463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.137733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915bd95-12c2-4767-bd5a-b0c745aa0b75 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.158040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28657e5-0ff9-4e6f-81a0-945289d144d1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.173128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.173920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.174206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.174484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.181766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.183220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.195402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.196965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.197443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:03.197553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.198542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:50:03.204430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:03.204975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.205058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 298/581] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.205058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.216373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:03.217582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:03.224670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.225632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:03.226498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 284/582] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.227332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.258204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.310570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193ee8-4359-41c6-a4d0-5e0749fcd3b2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.343447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bff60-9c13-4184-ad4d-10cf3a819db0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.348558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.349231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.349518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.349797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.350529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:03.350841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.352662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.364270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 299/583] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:03.364915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.386250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.387147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.387462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.387740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/54bb42878fff477099bbf56a46e9eec0/roles/4fc9795c5001472c86ae32739f03b3f6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.404957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=54bb42878fff477099bbf56a46e9eec0, role_id=4fc9795c5001472c86ae32739f03b3f6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.406435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.441446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.471357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71823c-5c66-4d91-9a9d-e9c040301f01 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.510964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 285/584] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:03 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/54bb42878fff477099bbf56a46e9eec0/roles/4fc9795c5001472c86ae32739f03b3f6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:03.511220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.511220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.511420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.511507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.511507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.511507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:03.511507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.511507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.534645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 300/585] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:03.535804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.612476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.646905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce196685-6a8c-48ff-a812-1504b537b5c0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.669066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b789b7-7117-4a3a-a19e-6a736ce5f1b9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.701068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.701742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.702026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.702307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.703058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:03.703386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.704762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.713734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.721266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 286/586] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:03.725685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.725685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.725685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.725685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.726077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.736392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.738178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:03.756870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.758074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:03.759828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:03.762300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 301/587] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.763392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.846019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dd317-6d34-4f53-8ca7-3f6e455d864d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.875429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8033-9896-4b10-8cae-acd58f414215 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:03.887623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.888693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.889124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.889577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.897118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:03.902349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.904078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.916162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:03.917201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:03.917686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:03.918129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:03.922856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.923961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.925012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:03.925012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:03.925012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.925012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:03.928888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 287/588] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.930812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:03.934630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:03.936383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:03.955514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:03.957063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:03.958262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:03.958262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:03.958262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:03.958262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:03.965232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 302/589] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:03.966396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.014754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.043372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b0fa1-72eb-4687-a200-9fa396888374 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.079102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eb289-aede-4771-b06b-36ac81b4ea47 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.081254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.082756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.082756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.082756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.083699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.084082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.085637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.106962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 288/590] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.107964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.118731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.119721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.120226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.120727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.121886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.122399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.124099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.137328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 303/591] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.138488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.169054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.225020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39829a50-2c54-44fe-86c4-58e73cca2eaf None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.251858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b79a7-baee-4a54-abde-d88ba67a92f2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.253842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.264327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.265047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.265281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.265560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.266293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.266678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.268000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.284864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 289/592] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.284864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.302084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.303049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.303366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.303648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.317484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.319556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:04.341709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.342632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.343566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:04.343566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:04.343566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:04.343566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:04.343566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 304/593] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.344287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.410635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44fa7-b137-44c8-af5c-7127e7268bba None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.421359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.447721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.448519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.448804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.449088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.455808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.457743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.458308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a94e1-b203-4931-9904-91abd9117ddf None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:04.473117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.474223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:04.475289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 290/594] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.480230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.495954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47d260-a9b4-45c1-b6b4-23750ba33435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.496857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d260-a9b4-45c1-b6b4-23750ba33435 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.497204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d260-a9b4-45c1-b6b4-23750ba33435 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.497547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d260-a9b4-45c1-b6b4-23750ba33435 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.505635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47d260-a9b4-45c1-b6b4-23750ba33435 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.507249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47d260-a9b4-45c1-b6b4-23750ba33435 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-dd47d260-a9b4-45c1-b6b4-23750ba33435 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:04.522524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.523698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:50:04.525039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:04.530414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 305/595] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.531637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadf5b6-95b0-4e85-917e-2635c3c877a2 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.532166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadf5b6-95b0-4e85-917e-2635c3c877a2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.532466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadf5b6-95b0-4e85-917e-2635c3c877a2 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.533931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 306/596] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:04.534507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadf5b6-95b0-4e85-917e-2635c3c877a2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.551309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.592891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a443171-8c3f-4358-911c-33a2711a2707 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.631840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.632511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.632788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.633064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.634081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.634389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.635786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.639062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadf5b6-95b0-4e85-917e-2635c3c877a2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.648115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 291/597] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.653719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.677265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ae3b6-5855-40ca-9425-c7bf3c802d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.677940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ae3b6-5855-40ca-9425-c7bf3c802d5c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.678228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ae3b6-5855-40ca-9425-c7bf3c802d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.678508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ae3b6-5855-40ca-9425-c7bf3c802d5c None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.679418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ae3b6-5855-40ca-9425-c7bf3c802d5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.679682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ae3b6-5855-40ca-9425-c7bf3c802d5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.681777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ae3b6-5855-40ca-9425-c7bf3c802d5c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.686037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.699704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 307/598] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.699704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecc233-2771-4dd4-bcae-e3dae2a788da None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.699704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecc233-2771-4dd4-bcae-e3dae2a788da None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.699704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecc233-2771-4dd4-bcae-e3dae2a788da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.715958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d7ecc233-2771-4dd4-bcae-e3dae2a788da None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:04.717158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d7ecc233-2771-4dd4-bcae-e3dae2a788da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:04.773732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131448d-82cd-4dc6-b2b1-72f110a940b1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.792096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 308/599] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:04.793301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecc233-2771-4dd4-bcae-e3dae2a788da None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.815866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.819921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.820680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.821679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.822209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.823100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:04.827194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.827194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.843633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 292/600] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:04.848797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:04.907688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecc233-2771-4dd4-bcae-e3dae2a788da None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.945058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:04.948483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:04.949415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:04.949715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:04.950059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:04.957126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:04.958521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:04.960012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747d274-25e1-4a6f-bd85-3ebd4eb9eb9f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:04.972659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:04.973541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:04.974457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 309/601] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:04.974944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.000946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.001834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.002122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.002398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/282c8662f2a2431b97d1bf122eabf5a7/roles/4fc9795c5001472c86ae32739f03b3f6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.017668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=282c8662f2a2431b97d1bf122eabf5a7, role_id=4fc9795c5001472c86ae32739f03b3f6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.019114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.041770 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.078937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f96f-dd7e-4fea-b1f3-eee736d2381b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.115219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 293/602] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:04 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/282c8662f2a2431b97d1bf122eabf5a7/roles/4fc9795c5001472c86ae32739f03b3f6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:05.115219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.129572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.131094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.131094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.131094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.144810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:05.153612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.154698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:05.155576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 310/603] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.156288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.279917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54624-a645-4f21-ba58-70884fff5c1d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.281529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0535e-c675-4536-896e-a15503137436 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.311070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.326024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.326697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.326982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.327295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.328093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.328275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.328414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.329405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.329821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.330088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.330555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.333842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.342380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.343848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.347679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 311/604] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.353559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:05.362333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.363260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.364182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:05.364182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:05.364182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:05.364182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:05.368592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 294/605] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.369480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.483333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b98fe1-a764-490f-8952-19b3d09cb2c0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.499772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf35e8-ae53-42dc-8554-b32a07eeec58 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.525585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.526419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.526711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.526995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.528114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.528430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.530255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.539273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.544758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 312/606] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.544758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.545144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.545549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.545650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/2678deaa97fc4012aa0d71f9e505f5ca/roles/4fc9795c5001472c86ae32739f03b3f6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.565588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=2678deaa97fc4012aa0d71f9e505f5ca, role_id=4fc9795c5001472c86ae32739f03b3f6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.567554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.599192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.649802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 295/607] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:05 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/2678deaa97fc4012aa0d71f9e505f5ca/roles/4fc9795c5001472c86ae32739f03b3f6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:05.651382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.660601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011bbb-d1fe-45b2-8549-3fbc96a8b58d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.702331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4a0bf-c5ff-4ed2-bf3c-a1b5d272ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.703370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4a0bf-c5ff-4ed2-bf3c-a1b5d272ba44 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.703842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4a0bf-c5ff-4ed2-bf3c-a1b5d272ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.704300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4a0bf-c5ff-4ed2-bf3c-a1b5d272ba44 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.705252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4a0bf-c5ff-4ed2-bf3c-a1b5d272ba44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:05.705813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4a0bf-c5ff-4ed2-bf3c-a1b5d272ba44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.707516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4a0bf-c5ff-4ed2-bf3c-a1b5d272ba44 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.710686 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.724303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 313/608] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:05.727782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8848-8140-4d57-ace2-45782adb0ada None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.727782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8848-8140-4d57-ace2-45782adb0ada None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.727782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8848-8140-4d57-ace2-45782adb0ada None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.729066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 314/609] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:05.737821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a8848-8140-4d57-ace2-45782adb0ada None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.800589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66b6cf-93ab-4b6d-8604-24c9ba3aa51a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.850108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33216b5a-335c-42cf-95dc-f7e619fa492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.850944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33216b5a-335c-42cf-95dc-f7e619fa492a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.852097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33216b5a-335c-42cf-95dc-f7e619fa492a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.852097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33216b5a-335c-42cf-95dc-f7e619fa492a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.859049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a8848-8140-4d57-ace2-45782adb0ada None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:05.863814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33216b5a-335c-42cf-95dc-f7e619fa492a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.865377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33216b5a-335c-42cf-95dc-f7e619fa492a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.870349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-33216b5a-335c-42cf-95dc-f7e619fa492a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:05.886409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.887428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.888337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:05.888337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:05.888337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:05.888337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:05.888337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 296/610] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.889125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6592-72d8-4315-8f1d-f79daa8412c2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.889417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6592-72d8-4315-8f1d-f79daa8412c2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.889591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6592-72d8-4315-8f1d-f79daa8412c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.908699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d23a6592-72d8-4315-8f1d-f79daa8412c2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:05.909898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d23a6592-72d8-4315-8f1d-f79daa8412c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:05.910571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:05.911697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:05.912505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:05.914037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:05.927206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:05.929448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:05.949971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:05.950901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:05.951859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:05.955002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 315/611] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:05.955984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:05.994626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 297/612] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:05.999364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6592-72d8-4315-8f1d-f79daa8412c2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.033761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.090243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662c3a5-864d-409e-8906-155cf8605dbe None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.128952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.129700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.130199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.130734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.131600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.132010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.133524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.142708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6592-72d8-4315-8f1d-f79daa8412c2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.151784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 316/613] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.152926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.180830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.181649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.182043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.182426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.183370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.183788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.185413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.196471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.199862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 298/614] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.203770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.298394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d30965-d275-4754-9c67-3dedca6a924d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.314055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.338011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848b5bb-4110-4f57-a29b-146460295b2b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.345345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.346784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.347297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.348089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.363782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:06.373729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:06.374899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:06.375977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:06.382134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 317/615] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:06.382134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.389014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.390018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.390508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.390954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.399758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.401561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:06.418078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:06.419299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:06.420472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:06.422897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 299/616] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:06.423880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.475124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.491006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385e927-1619-415e-a391-1edbcddb1b01 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.527590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.528499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.528870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.529280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/cf08ecfc3fed44e2bc1ccff8521a4ed8/roles/4fc9795c5001472c86ae32739f03b3f6` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.537707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eff32-7aec-4145-9f50-81041cd7f284 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.544830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=cf08ecfc3fed44e2bc1ccff8521a4ed8, role_id=4fc9795c5001472c86ae32739f03b3f6)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.546470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.574320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.575066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.575398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.575680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.576432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.576744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.578100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.587162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 300/617] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.587742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.594600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.635909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 318/618] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:06 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/cf08ecfc3fed44e2bc1ccff8521a4ed8/roles/4fc9795c5001472c86ae32739f03b3f6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:06.636365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.760454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687683d4-fb41-45ef-abb1-38ad6bd7963e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.818358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.819048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.819364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.819647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.821784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:06.822042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.823538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.852561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9cdb-240f-47a1-84ed-fd272f5df1f9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:06.857355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 301/619] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:06.861963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:06.900468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:06.901205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:06.901490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:06.901778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:06.905268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.914183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:06.919991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:06.936685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:06.938006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:06.939067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:06.939067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:06.939067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:06.939067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:06.939067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 319/620] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:06.943401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:06.943533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.022737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa9949-3257-45ad-914c-d3877735c7af None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.092520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.093571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.094029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.094454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.112941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.115582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:07.137560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:07.138699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:07.139784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:07.139784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:07.139784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:07.139784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:07.141806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 302/621] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:07.146297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b1377-cf53-4cd3-b02d-88380b943985 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.148589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.213305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fde2f7-95ed-4775-bdde-8b9c1785c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.214343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde2f7-95ed-4775-bdde-8b9c1785c0dd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.215543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde2f7-95ed-4775-bdde-8b9c1785c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.216137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde2f7-95ed-4775-bdde-8b9c1785c0dd None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.217041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fde2f7-95ed-4775-bdde-8b9c1785c0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:07.221031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fde2f7-95ed-4775-bdde-8b9c1785c0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.222971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fde2f7-95ed-4775-bdde-8b9c1785c0dd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.239987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 320/622] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:07.241781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af808-05a7-44c1-a693-957c836ee621 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.242282 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af808-05a7-44c1-a693-957c836ee621 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.244094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af808-05a7-44c1-a693-957c836ee621 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.249782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 321/623] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:07.250960 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217af808-05a7-44c1-a693-957c836ee621 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.299286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cef8e-57ce-473e-b50e-4532b6fa1019 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.310357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.317774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.343749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fdeec-bba6-4899-b399-b31245f12f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.345417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fdeec-bba6-4899-b399-b31245f12f84 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.345707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fdeec-bba6-4899-b399-b31245f12f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.346609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fdeec-bba6-4899-b399-b31245f12f84 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.347903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fdeec-bba6-4899-b399-b31245f12f84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:07.348222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fdeec-bba6-4899-b399-b31245f12f84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.350396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fdeec-bba6-4899-b399-b31245f12f84 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.371393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 303/624] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:07.373593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4003bc-1bfd-4dde-99e0-a1be835e54c7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.373899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4003bc-1bfd-4dde-99e0-a1be835e54c7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.375552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4003bc-1bfd-4dde-99e0-a1be835e54c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.377822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217af808-05a7-44c1-a693-957c836ee621 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.396355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ed4003bc-1bfd-4dde-99e0-a1be835e54c7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:07.398084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ed4003bc-1bfd-4dde-99e0-a1be835e54c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:07.421634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.422534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.423048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.433309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.435037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:07.459358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:07.460568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:07.462314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:07.464361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 322/625] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:07.465521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.490259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 304/626] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:07.491441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4003bc-1bfd-4dde-99e0-a1be835e54c7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.597242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef03c1b-c2cc-4509-a0a7-7a8b0585c2be None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.609973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.622856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4003bc-1bfd-4dde-99e0-a1be835e54c7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.625115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.641778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.644185 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.644640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.645078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.652981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.654598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.662168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.662991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.663409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.663792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.664933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:07.665352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.666879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:07.670585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:07.671792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:07.672789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:07.679547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 323/627] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:07.681272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.687271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 305/628] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:07.688277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.794120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2887-2d52-409e-853a-679877d828b5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.795189 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0a2c9-313c-48ae-9850-306fdd1d9b30 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.831789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.832409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:07.833104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.833235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:07.833506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.833855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.834027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:07.834468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:07.834829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:07.835224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:07.837573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:07.848037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:07.855533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 306/629] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:07.855533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:07.861826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:07.863437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:07.864319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:07.866708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 324/630] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:07.868736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:07.875756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.901456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:07.966111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fbec2-9004-4411-a8e9-ac3fc61c2eee None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:07.998260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ca886-1e0d-4ea1-90e2-7a68e2fba1f7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.003229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b4f1c-d906-4d18-8c1f-fc2890de2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.003890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b4f1c-d906-4d18-8c1f-fc2890de2dae None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.004179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b4f1c-d906-4d18-8c1f-fc2890de2dae None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.004458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b4f1c-d906-4d18-8c1f-fc2890de2dae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.036240 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.037135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.037423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.037702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/f766fee8481a4eccbc56d4af0c2afbf2/roles/4fc9795c5001472c86ae32739f03b3f6` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.054523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=f766fee8481a4eccbc56d4af0c2afbf2, role_id=4fc9795c5001472c86ae32739f03b3f6)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.057507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.081041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b4f1c-d906-4d18-8c1f-fc2890de2dae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.082486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b4f1c-d906-4d18-8c1f-fc2890de2dae None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.103455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.135091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 325/631] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:07 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/f766fee8481a4eccbc56d4af0c2afbf2/roles/4fc9795c5001472c86ae32739f03b3f6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:08.136173 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.205228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 307/632] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:08.207508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375fa37-98b6-4eee-aedd-685c6ebe9783 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.208099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375fa37-98b6-4eee-aedd-685c6ebe9783 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.208620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375fa37-98b6-4eee-aedd-685c6ebe9783 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.210839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 308/633] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:08.212118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375fa37-98b6-4eee-aedd-685c6ebe9783 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.264865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.297081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb5eaf-06cf-4eb3-b52f-86ff51bfbdef None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.340516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375fa37-98b6-4eee-aedd-685c6ebe9783 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.356447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea9a5c-3c1a-4e14-b4ac-367ff9bc2de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.357262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea9a5c-3c1a-4e14-b4ac-367ff9bc2de6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.357667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea9a5c-3c1a-4e14-b4ac-367ff9bc2de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.358038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea9a5c-3c1a-4e14-b4ac-367ff9bc2de6 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.359215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea9a5c-3c1a-4e14-b4ac-367ff9bc2de6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:08.359442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea9a5c-3c1a-4e14-b4ac-367ff9bc2de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.360934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea9a5c-3c1a-4e14-b4ac-367ff9bc2de6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.365655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.374332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 326/634] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:08.378965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6de33-6fcf-4ebf-a30a-64cf9dc922fd None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.378965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6de33-6fcf-4ebf-a30a-64cf9dc922fd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.378965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6de33-6fcf-4ebf-a30a-64cf9dc922fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.382288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.383872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.384299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.384754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.395835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.397538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.397885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f0d6de33-6fcf-4ebf-a30a-64cf9dc922fd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:08.399752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f0d6de33-6fcf-4ebf-a30a-64cf9dc922fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:08.416827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:08.417944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:08.419049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:08.419049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:08.419049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:08.419049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:08.420573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 309/635] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:08.421471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.483077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 327/636] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:08.484411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6de33-6fcf-4ebf-a30a-64cf9dc922fd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.532438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93fce5-cd08-4aab-9726-b9db5691a46a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.562096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.570584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.571922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.572476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.583731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.584048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.585673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.598049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6de33-6fcf-4ebf-a30a-64cf9dc922fd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:08.606029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:08.607133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:08.608265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:08.608265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:08.608265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:08.608265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:08.609570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 310/637] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:08.610485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.637277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.640432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.641313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.641748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.642166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.643054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:08.643535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.645081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.658913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 328/638] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:08.660077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.725944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324ec0a-f3d5-411f-a88c-1b5284003613 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.762870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.763821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.764118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.764403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.773585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7005ac-fca3-4852-92ee-6fa7d6de0395 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.776180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:08.785033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:08.786165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:08.787261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:08.791126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 311/639] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:08.792021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.808337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.814924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.815802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.816258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.816695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.817634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:08.818107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.819659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.832866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 329/640] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:08.833858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:08.901198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:08.904969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda7cca-28d1-4949-8aae-812c65340bea None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.947636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:08.948691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:08.949132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:08.949438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:08.956789 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:08.958236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:08.967891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1d3b3-ccad-468c-b699-525ffbd7c064 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:08.972975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:08.974144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 312/641] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:08.980106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.009296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4980b5c-629f-4f8d-9bff-aec5e2f54eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.010229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980b5c-629f-4f8d-9bff-aec5e2f54eeb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.010688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980b5c-629f-4f8d-9bff-aec5e2f54eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.010972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4980b5c-629f-4f8d-9bff-aec5e2f54eeb None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.011760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980b5c-629f-4f8d-9bff-aec5e2f54eeb None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:09.012072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980b5c-629f-4f8d-9bff-aec5e2f54eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.013447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4980b5c-629f-4f8d-9bff-aec5e2f54eeb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.022019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 330/642] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:09.023074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258db22-42f4-40a7-a839-9311acfd634c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.023455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258db22-42f4-40a7-a839-9311acfd634c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.023972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258db22-42f4-40a7-a839-9311acfd634c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.025335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 331/643] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:09.028241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258db22-42f4-40a7-a839-9311acfd634c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.085285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821f78c-e754-4dc9-8d59-48d2d531a5d6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.091694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.122426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.123256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.123549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.123831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.131417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.132849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.141897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258db22-42f4-40a7-a839-9311acfd634c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:09.147607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:09.148581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 10:50:09.150239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:09.153947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 313/644] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:09.154577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.158910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.181701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d73378-880b-4da4-a3df-26c69b93bde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.182368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d73378-880b-4da4-a3df-26c69b93bde0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.182656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d73378-880b-4da4-a3df-26c69b93bde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.182937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d73378-880b-4da4-a3df-26c69b93bde0 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.183773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d73378-880b-4da4-a3df-26c69b93bde0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:09.184089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d73378-880b-4da4-a3df-26c69b93bde0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.185534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d73378-880b-4da4-a3df-26c69b93bde0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.203087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 332/645] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:09.203087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2ed49-075c-4bcf-abea-d3df431e43c6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.203087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2ed49-075c-4bcf-abea-d3df431e43c6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.203087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2ed49-075c-4bcf-abea-d3df431e43c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.220279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1da2ed49-075c-4bcf-abea-d3df431e43c6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:09.221602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1da2ed49-075c-4bcf-abea-d3df431e43c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:09.263540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8766f-5ddc-4da9-a47b-c9472fa16462 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.296278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.297073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.297362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.297466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 333/646] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:09.297868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.298894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2ed49-075c-4bcf-abea-d3df431e43c6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.324387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.325841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:09.343622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:09.344747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:09.345856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:09.345856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:09.345856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:09.345856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:09.351550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 314/647] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:09.351550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.363893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.411612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.427543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2ed49-075c-4bcf-abea-d3df431e43c6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.456140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cce32-b5fe-4fab-83fc-ba2959a18d38 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.472690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.473646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.473942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.474227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.475080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:09.475425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.478460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.490202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.490871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.491161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.491485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.492321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=54bb42878fff477099bbf56a46e9eec0) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:09.492886 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.494575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.501556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 334/648] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:09.501556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.501776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 315/649] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/role_assignments?user.id=54bb42878fff477099bbf56a46e9eec0&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:09.501776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.624987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52eb80-b362-4f42-9412-1d926e4845ab None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.648394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.653026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.656332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5616f0f-b7d2-4b22-9caa-1020da400019 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.669417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.670364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.670853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.671353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.672268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:09.672790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.674339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.691811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 316/650] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:09.692927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.701972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.703441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.703441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.703441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.717886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:09.726033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:09.727833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:09.729447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 335/651] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:09.732920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.847071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11734806-b9b5-422b-b1e1-4993684092fc None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.847743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65089540-427a-46d5-bb49-8e125772f410 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:09.885455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.886476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.886799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.887201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.895670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:09.896690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.903322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:09.903322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:09.903322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:09.903611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:09.913860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:09.915190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:09.916372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:09.920073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 336/652] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:09.920073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:09.948034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.968479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:09.992487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:09.994212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.047374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc38810-a226-4c1c-a783-51381dfd9e43 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.094920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.096164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.096599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.097536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.098955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.099565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.100954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.111227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 317/653] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:10.111227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.116487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 337/654] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.118018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.228616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.229557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278dbe-4ce4-42e0-85cc-cc2ea5c2ae98 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.231620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.254884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2acc66-8ede-461f-bc10-e2565d407454 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.279043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.281149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.281819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.282359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.283640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.284514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.286627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.293922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.294895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.295337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.295715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.307508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.309088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.311437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 318/655] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.312867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:10.328988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.329949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.330863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:10.330863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:10.330863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.330863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:10.330863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 338/656] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.331145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.423112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9f109-5966-461d-bbbd-799847273417 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.437266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb96de-a9ea-4965-b2c7-99587b97f037 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.457371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fed0d-0277-41f1-be0a-5e5d5806ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.458164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fed0d-0277-41f1-be0a-5e5d5806ed11 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.458452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fed0d-0277-41f1-be0a-5e5d5806ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.458731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fed0d-0277-41f1-be0a-5e5d5806ed11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.465399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.469262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fed0d-0277-41f1-be0a-5e5d5806ed11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.471037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fed0d-0277-41f1-be0a-5e5d5806ed11 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.475407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4286d-d4a0-4eed-8d61-90f6113cb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.476220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4286d-d4a0-4eed-8d61-90f6113cb989 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.477183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4286d-d4a0-4eed-8d61-90f6113cb989 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.480951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4286d-d4a0-4eed-8d61-90f6113cb989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.480951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-cc1fed0d-0277-41f1-be0a-5e5d5806ed11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:10.490559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.491673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.492769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:10.492769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:10.492769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:10.492769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:10.492769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 319/657] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.493399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb6c5b-016e-41cf-a088-d449bfd2cc6d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.493661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb6c5b-016e-41cf-a088-d449bfd2cc6d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.493961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb6c5b-016e-41cf-a088-d449bfd2cc6d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.499127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 320/658] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:10.499127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb6c5b-016e-41cf-a088-d449bfd2cc6d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.560320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4286d-d4a0-4eed-8d61-90f6113cb989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.561798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4286d-d4a0-4eed-8d61-90f6113cb989 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.602014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb6c5b-016e-41cf-a088-d449bfd2cc6d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.639073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.639767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.640058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.640342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.641074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.641390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.642754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.654890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 321/659] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.655546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.676605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 339/660] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:10.677831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9692-d63a-45c4-8cc4-eb80f31493c9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.678132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9692-d63a-45c4-8cc4-eb80f31493c9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.678411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9692-d63a-45c4-8cc4-eb80f31493c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.695762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ddbc9692-d63a-45c4-8cc4-eb80f31493c9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:10.696938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ddbc9692-d63a-45c4-8cc4-eb80f31493c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:10.706176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.725711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.764453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd781431-1bf0-4149-b035-9bf96d33007d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.779205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 340/661] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:10.779205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc9692-d63a-45c4-8cc4-eb80f31493c9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.808816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.809901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.809901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.810111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.810762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:10.811080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.812492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.825520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 322/662] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:10.826350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.882696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc9692-d63a-45c4-8cc4-eb80f31493c9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.921884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.922696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.922981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.923294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:10.930333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:10.932338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:10.946014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:10.946916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:10.947887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:10.948324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389a94a-f1ec-4262-8fa3-d3e22ece148f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:10.948411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 341/663] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:10.948411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:10.962676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.974898 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:10.988571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:10.989374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:10.989662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:10.989941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.001124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.002602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:11.022353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.023393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.024514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:11.024514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:11.024514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:11.024514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:11.024514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 323/664] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.024514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.063886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93164d45-08b8-40c4-9b0c-13c0eb1705c0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.100768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.101567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.102437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.102722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.112905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:11.121545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.122477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:11.123360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 342/665] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.124087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.135228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada10fd8-b5f3-4243-9e4a-83843837cb42 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.171382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.172481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.172953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.173430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.180998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.182515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:11.198103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.199088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:11.200372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 324/666] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.201003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.218079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.227995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.235443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43010ef-3ef0-43bb-b425-bcc54fd9fccc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.275042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.275670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.275961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.276243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.277047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.277362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.278809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.294415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 343/667] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.295581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.308884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618a9f8-ac0e-4e0c-928d-3c6b414a3c1f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.349357 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.350414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.350706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.350989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.352385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.352761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.354649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.369864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 325/668] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.370813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.404328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef03fbc-53ac-43be-9276-98b322efff46 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.440516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.441339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.441453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.441731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.442478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.442785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.444279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.453546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 344/669] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.454165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.462292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.465509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.487560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5386cd-8985-4afa-8b1a-709b0180eaad None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.525128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.525725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.526052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.526337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.527172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.527509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.528936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.550559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 326/670] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.550559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.562758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784c3e3-facf-4cd2-88b2-85f392422bde None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.598161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.598883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.599197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.599494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.600352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=282c8662f2a2431b97d1bf122eabf5a7) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.600673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.602204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.608239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 345/671] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/role_assignments?user.id=282c8662f2a2431b97d1bf122eabf5a7&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.609036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.688043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99dd1c-7ed9-4814-9954-44ab0643965a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.688879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.722973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988be5df-33e2-404e-8965-54053fc3d322 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.735137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.736018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.736307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.736588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.744973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.746447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.749325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.761762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae084569-93e0-445b-b756-d7703616a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.762571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae084569-93e0-445b-b756-d7703616a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:11.762746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.763689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:11.764662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:11.765564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae084569-93e0-445b-b756-d7703616a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.765564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae084569-93e0-445b-b756-d7703616a9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.768851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 327/672] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.769776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.777689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae084569-93e0-445b-b756-d7703616a9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.779368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae084569-93e0-445b-b756-d7703616a9d4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ae084569-93e0-445b-b756-d7703616a9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:11.803088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:11.804059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 346/673] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ef56-3b07-49f7-94d3-ee3474ebb7e9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ef56-3b07-49f7-94d3-ee3474ebb7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.807736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ef56-3b07-49f7-94d3-ee3474ebb7e9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.811217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 347/674] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:11.811217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ef56-3b07-49f7-94d3-ee3474ebb7e9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:11.892157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72295b-20d8-45c8-bc61-bead09332fe7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.928344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ef56-3b07-49f7-94d3-ee3474ebb7e9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:11.937703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70861b-371a-4387-bf3e-b6e42970004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.938713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70861b-371a-4387-bf3e-b6e42970004d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.939004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70861b-371a-4387-bf3e-b6e42970004d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.939322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70861b-371a-4387-bf3e-b6e42970004d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.956678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:11.970224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:11.970891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:11.971203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:11.971701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:11.972629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:11.972949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:11.974389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:11.992360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 348/675] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:11.993749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.008679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.011558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70861b-371a-4387-bf3e-b6e42970004d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.013085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70861b-371a-4387-bf3e-b6e42970004d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.106667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c408-b9c0-4864-8b0f-232a07d4f51c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.124610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 328/676] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:12.125910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0cdaa-258a-4968-9578-e3ce5fa39b41 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.126202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0cdaa-258a-4968-9578-e3ce5fa39b41 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.126550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0cdaa-258a-4968-9578-e3ce5fa39b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.142540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3ce0cdaa-258a-4968-9578-e3ce5fa39b41 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:12.143768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3ce0cdaa-258a-4968-9578-e3ce5fa39b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:12.147371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.148276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.149355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.149990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.150975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.151512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.153089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.165599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 349/677] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.166423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.209409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.231311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 329/678] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:12.231311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0cdaa-258a-4968-9578-e3ce5fa39b41 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.264706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.275084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160514-58dc-46d4-8efd-1c173a0ded78 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.311420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.312079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.312369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.312655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.313494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=2678deaa97fc4012aa0d71f9e505f5ca) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.313811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.315301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.321441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 350/679] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/role_assignments?user.id=2678deaa97fc4012aa0d71f9e505f5ca&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.328994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.344978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0cdaa-258a-4968-9578-e3ce5fa39b41 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.395043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.396497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.397067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.397722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.413495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.416595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:12.438340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:12.439362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:12.440278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:12.440278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:12.440278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:12.440278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:12.443861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 330/680] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:12.445299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.471210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193558c-c20d-4c6a-993e-5a2c07912318 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.481111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.520278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.521162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.521452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.521724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.530819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.532315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:12.548978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:12.550167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:12.551370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 351/681] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:12.552003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.553437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.572813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31852-3be6-412c-bd18-bec63c4b3e09 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.609546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.610215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.610503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.610785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.622358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:12.630096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:12.631063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:12.632044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 331/682] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:12.632756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.666262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81587cb8-590f-4a09-9d98-88dde20a55d1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.701492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.702430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.702858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.703299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.704192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.704648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.706176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.718806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 352/683] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.719868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.740326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad93192-9d9e-4492-8c65-7601e26c8f69 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.751526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.780410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.781103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.781392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.781672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.782665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.783254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.784740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.800497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:12.805234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 332/684] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.806220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.831394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6e219-3001-4d92-a4ee-9726871a12aa None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.871568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.872735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.873225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.873819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.874834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:12.875402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.877175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.887572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 353/685] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:12.888726 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:12.939103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f9f3f-79e2-423b-92d6-bdbe3cbe79de None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:12.975825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:12.976919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:12.977430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:12.977875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:12.985413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:12.987069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:12.995521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:13.002009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:13.003077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:13.004115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:13.005934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 333/686] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:13.006850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.011856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46004d3d-dccb-4595-b5be-211959e2e256 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.057055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.058627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.059149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.059533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.090132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.141012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.142150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602ad4-99e4-47f3-9d02-ede050b707d6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.142897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.185448 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.186697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.186945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.187354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.199674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.201192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:13.220417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:13.221445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:13.222620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:13.222620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:13.222620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:13.222620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:13.222842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 334/687] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:13.224132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.242151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.256502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 354/688] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:13.257450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.334800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c8d65-7cfd-4534-82aa-19aacd0fd6e3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.349123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.366090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2013e38-9634-4df3-93dc-419a095d76ec None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.373024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.374029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.374513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.374997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.376003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.376487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.378270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.399465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 335/689] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:13.400610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.406578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.407340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.407631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.407912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.408652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.408964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.410332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.423871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 355/690] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:13.424784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.480230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.509752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db625978-0638-4782-83b0-3e6d403efd44 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.537546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5942d-0ced-4996-8e93-28cc6c5f23b6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.545442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.546511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.546996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.547513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.558650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.560583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.573593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.574259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.574547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.574830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.575703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=cf08ecfc3fed44e2bc1ccff8521a4ed8) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.576022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.577528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:13.580944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:13.581911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:13.582834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:13.582834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:13.582834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:13.582834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:13.591218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 356/691] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/role_assignments?user.id=cf08ecfc3fed44e2bc1ccff8521a4ed8&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:13.591218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.591481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 336/692] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:13.597839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.649010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.792097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228f86-7d84-4fde-97b7-f7e28a10c974 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.807093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf42d-a4e1-4a6b-8e72-49cf826e8c5c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:13.829635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:13.844127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233655a0-3cef-4c96-9a7b-952f7eb9c7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.845141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233655a0-3cef-4c96-9a7b-952f7eb9c7db None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.845629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233655a0-3cef-4c96-9a7b-952f7eb9c7db None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.846104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233655a0-3cef-4c96-9a7b-952f7eb9c7db None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.847345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233655a0-3cef-4c96-9a7b-952f7eb9c7db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:13.847856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233655a0-3cef-4c96-9a7b-952f7eb9c7db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.849859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233655a0-3cef-4c96-9a7b-952f7eb9c7db None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.856798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabcb85-8afb-4f2f-868e-ba36b3d2e119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:13.858056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabcb85-8afb-4f2f-868e-ba36b3d2e119 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.858056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabcb85-8afb-4f2f-868e-ba36b3d2e119 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.858316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabcb85-8afb-4f2f-868e-ba36b3d2e119 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.863873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 337/693] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:13.865914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60412a6-d9f3-4997-9788-549d2b951a45 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.866485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60412a6-d9f3-4997-9788-549d2b951a45 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.866607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabcb85-8afb-4f2f-868e-ba36b3d2e119 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:13.867781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60412a6-d9f3-4997-9788-549d2b951a45 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.868690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabcb85-8afb-4f2f-868e-ba36b3d2e119 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:13.870138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 338/694] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:13.871165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60412a6-d9f3-4997-9788-549d2b951a45 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-5fabcb85-8afb-4f2f-868e-ba36b3d2e119 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:13.885760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:13.886763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:13.887846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 357/695] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:13.889060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e407c8-11f7-4696-b188-1b9c28edb1b4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:13.889060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e407c8-11f7-4696-b188-1b9c28edb1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:13.889228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e407c8-11f7-4696-b188-1b9c28edb1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:13.911388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d7e407c8-11f7-4696-b188-1b9c28edb1b4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:13.917102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d7e407c8-11f7-4696-b188-1b9c28edb1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:13.978354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.005264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60412a6-d9f3-4997-9788-549d2b951a45 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.011310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 358/696] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:14.012371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e407c8-11f7-4696-b188-1b9c28edb1b4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.043439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.044274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.044631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.045001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.052236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.053776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:14.068576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:14.069510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:14.071013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 339/697] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:14.071536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.143424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e407c8-11f7-4696-b188-1b9c28edb1b4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.145371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.183471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.184298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.184779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.185178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.186288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.186714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.188432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.192972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f8526-c8c0-4180-a1b7-3b506ad6d6aa None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.209021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 359/698] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:14.209953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.235274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.236089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.236379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.236665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.248048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.249685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.252780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:14.269483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:14.270609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:14.271651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:14.271651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:14.271651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:14.271651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:14.273507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 340/699] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:14.274955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.337556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f066c-7fd0-48f0-b535-8517db8aff83 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.376071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.376762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.377053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.377337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.378235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.378552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.380034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.390974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47af93-13ca-48a0-ae87-fb6ae9d24a06 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.399137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 360/700] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:14.400725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.425171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.430357 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.431293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.431720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.432165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.501616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.513340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.516186 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2268fc-d0ba-4f96-8a98-58fc3af5c726 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.517510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.556448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.557042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.557337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.557620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.558356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.558668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.560074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.578105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 361/701] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:14.579083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.632888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 341/702] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:14.634081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.676189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.701670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598e0c2-2f0f-4770-a489-a80f22fff3b8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.733233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.740126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7368db-053d-4cd0-8999-161675fc1305 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.742056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.743321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.743717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.744039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.744921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=f766fee8481a4eccbc56d4af0c2afbf2) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:14.745291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.746754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.753348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 362/703] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/role_assignments?user.id=f766fee8481a4eccbc56d4af0c2afbf2&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:14.754022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:14.798779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3fd03-11c0-444f-a265-7b223a2d7c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.799502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3fd03-11c0-444f-a265-7b223a2d7c85 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.799787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3fd03-11c0-444f-a265-7b223a2d7c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.800065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3fd03-11c0-444f-a265-7b223a2d7c85 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.801195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3fd03-11c0-444f-a265-7b223a2d7c85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.802255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3fd03-11c0-444f-a265-7b223a2d7c85 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.887908 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0303-1c58-4aa3-8f74-3210ef9d7c20 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:14.927135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 342/704] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:50:14 2022] POST /identity/v3/users => generated 254 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:14.928553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97722-ff51-40b2-8b8d-eed1fcef3eda None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.928848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97722-ff51-40b2-8b8d-eed1fcef3eda None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.929451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97722-ff51-40b2-8b8d-eed1fcef3eda None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.931221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 343/705] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:14.936655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14278d8-8941-4856-9668-4ac6975d6c70 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.937310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14278d8-8941-4856-9668-4ac6975d6c70 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.937895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14278d8-8941-4856-9668-4ac6975d6c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.941835 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:14.942927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:14.943431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:14.943943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:14.957857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:14.959713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:14.960533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e14278d8-8941-4856-9668-4ac6975d6c70 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:14.962738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e14278d8-8941-4856-9668-4ac6975d6c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:14.965057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:14.983858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:14.996300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:14.997441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:14.997441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:14.997441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:14.997441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:14.997441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 363/706] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:14.997441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.049364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 344/707] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:15.073387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14278d8-8941-4856-9668-4ac6975d6c70 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.130543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69229343-8536-4a53-856c-41c2d78c3d66 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.169159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.173885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.174722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.175135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.175570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.176581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:15.176896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.178279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.192685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14278d8-8941-4856-9668-4ac6975d6c70 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.197442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 364/708] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:15.206905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.236362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.237454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.237745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.238024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.238685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.240373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:15.260085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:15.261146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:15.262022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:15.265140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 345/709] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:15.271386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.283524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.343530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6edd73-e7b8-497c-b8a1-503153e4c983 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.387678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.388711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.389380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.389761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.402832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4810f-04e1-4c87-8582-d816ca1350e8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.405112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.406582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:15.429176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:15.430236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:15.431148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:15.431621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 365/710] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:15.444217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.462329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.463014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.463338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.463617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.464495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:15.464762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.466150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.475368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 346/711] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:15.478070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.507588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.585675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a7bf5-91d6-4112-8e2c-08c999113ab7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.623530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.624215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.624503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.624785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.625491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:15.625804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.627192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.635379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 347/712] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:15.669741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c20ae5-1820-4cce-9ac5-9ecf32653b22 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.696584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.722753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c387f1-a2d2-4dd9-9ffe-14b4b8f4a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.723475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c387f1-a2d2-4dd9-9ffe-14b4b8f4a17e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.723766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c387f1-a2d2-4dd9-9ffe-14b4b8f4a17e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.724049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c387f1-a2d2-4dd9-9ffe-14b4b8f4a17e None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.724872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c387f1-a2d2-4dd9-9ffe-14b4b8f4a17e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:15.725191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c387f1-a2d2-4dd9-9ffe-14b4b8f4a17e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.727379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c387f1-a2d2-4dd9-9ffe-14b4b8f4a17e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.749769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 366/713] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:15.755717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:15.867896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:15.912414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db22fbb-5e12-4967-a692-029cadb3e56f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:15.960599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd56dc-3a21-4352-b40a-7cf04e5ebbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:15.961226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd56dc-3a21-4352-b40a-7cf04e5ebbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.961522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd56dc-3a21-4352-b40a-7cf04e5ebbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.961788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd56dc-3a21-4352-b40a-7cf04e5ebbf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.962636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd56dc-3a21-4352-b40a-7cf04e5ebbf3 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=54bb42878fff477099bbf56a46e9eec0) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:15.962954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd56dc-3a21-4352-b40a-7cf04e5ebbf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:15.964422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd56dc-3a21-4352-b40a-7cf04e5ebbf3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:15.970755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 348/714] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:15 2022] GET /identity/v3/role_assignments?user.id=54bb42878fff477099bbf56a46e9eec0&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:15.974256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9b727-4bb9-4524-a691-a309ab95c871 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.974550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9b727-4bb9-4524-a691-a309ab95c871 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.974833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9b727-4bb9-4524-a691-a309ab95c871 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:15.976561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 367/715] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:15.982421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ac8bd-33a2-4c96-9e0c-644c22096c15 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:15.982752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ac8bd-33a2-4c96-9e0c-644c22096c15 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:15.982982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ac8bd-33a2-4c96-9e0c-644c22096c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.001315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-228ac8bd-33a2-4c96-9e0c-644c22096c15 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:16.002532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-228ac8bd-33a2-4c96-9e0c-644c22096c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:16.098608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 349/716] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:16.128822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9b727-4bb9-4524-a691-a309ab95c871 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.247287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9b727-4bb9-4524-a691-a309ab95c871 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.287065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.288243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.288754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.289879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] PATH_INFO: `/v3/services/network` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.290815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.292437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:16.306761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:16.307868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:16.308721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:16.311527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 368/717] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:16.317581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ac8bd-33a2-4c96-9e0c-644c22096c15 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.405438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.458341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ac8bd-33a2-4c96-9e0c-644c22096c15 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.503203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df34b18-0c2c-4a40-b814-bdd8fa7155a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.504222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df34b18-0c2c-4a40-b814-bdd8fa7155a4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.504708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df34b18-0c2c-4a40-b814-bdd8fa7155a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.505212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df34b18-0c2c-4a40-b814-bdd8fa7155a4 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.506184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df34b18-0c2c-4a40-b814-bdd8fa7155a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:16.506691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df34b18-0c2c-4a40-b814-bdd8fa7155a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.508850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df34b18-0c2c-4a40-b814-bdd8fa7155a4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.518927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 350/718] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:16.524677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:16.573254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:16.655964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c208368-1baa-44ac-ad15-e36434913c58 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:16.705727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbe205-976b-4d37-be70-2aaa81b9f5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:16.707172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbe205-976b-4d37-be70-2aaa81b9f5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:16.707721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbe205-976b-4d37-be70-2aaa81b9f5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:16.708197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbe205-976b-4d37-be70-2aaa81b9f5e3 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:16.709109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbe205-976b-4d37-be70-2aaa81b9f5e3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:16.709616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbe205-976b-4d37-be70-2aaa81b9f5e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:16.711806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbe205-976b-4d37-be70-2aaa81b9f5e3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:16.722694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 369/719] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:17.493735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe68c6-c664-4638-bb4d-56583afdc7d8 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.494262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe68c6-c664-4638-bb4d-56583afdc7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.494341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe68c6-c664-4638-bb4d-56583afdc7d8 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.495961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 351/720] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:17.501048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb99a73-7786-490a-bd61-ce9c9fd1714f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.501345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb99a73-7786-490a-bd61-ce9c9fd1714f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.501625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb99a73-7786-490a-bd61-ce9c9fd1714f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.523867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7eb99a73-7786-490a-bd61-ce9c9fd1714f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:17.525289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7eb99a73-7786-490a-bd61-ce9c9fd1714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:17.601921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 370/721] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:17.650007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe68c6-c664-4638-bb4d-56583afdc7d8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:17.667852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ab4ff-1c62-4e9b-bb76-efda2dbe8d1e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.668133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ab4ff-1c62-4e9b-bb76-efda2dbe8d1e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.668415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ab4ff-1c62-4e9b-bb76-efda2dbe8d1e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.670092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 371/722] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:17.675151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51634a3-5810-45a7-ac02-c6fd4eb25c75 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.675472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51634a3-5810-45a7-ac02-c6fd4eb25c75 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.675752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51634a3-5810-45a7-ac02-c6fd4eb25c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.693453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b51634a3-5810-45a7-ac02-c6fd4eb25c75 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:17.694640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b51634a3-5810-45a7-ac02-c6fd4eb25c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:17.734624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:17.809495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 372/723] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 10:50:17.837042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51634a3-5810-45a7-ac02-c6fd4eb25c75 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:17.854035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe68c6-c664-4638-bb4d-56583afdc7d8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:17.910731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:17.917213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:17.918317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:17.918422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:17.918668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:17.943817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:17.959988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:17.961145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:17.962169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:17.965888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 352/724] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:17.972560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.031690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51634a3-5810-45a7-ac02-c6fd4eb25c75 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.097777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.098586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.098870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.099145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.120518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:18.129003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:18.130728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:18.132596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:18.138382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 373/725] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:18.140278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.164880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846b3e4-a864-4877-ad1e-f018e35a333a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.191810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.229598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.230283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.230573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.230857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.231766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:18.232088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.233580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.243996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 353/726] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:18.256158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.323019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf155bb0-a588-4a68-9c51-202cf72b72ca None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.379629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.394699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c527942-0864-4023-8aa8-53074622fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.395456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c527942-0864-4023-8aa8-53074622fee2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.395742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c527942-0864-4023-8aa8-53074622fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.396021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c527942-0864-4023-8aa8-53074622fee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.396781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c527942-0864-4023-8aa8-53074622fee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:18.397092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c527942-0864-4023-8aa8-53074622fee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.399113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c527942-0864-4023-8aa8-53074622fee2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.422624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 374/727] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:18.443834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f0301-c8ed-4192-98f6-f7fc4d5c8100 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.444124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f0301-c8ed-4192-98f6-f7fc4d5c8100 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.444329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541820f-b820-42f8-a56c-52beccb9458a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.444445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f0301-c8ed-4192-98f6-f7fc4d5c8100 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.447099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 375/728] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:18.448666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f0301-c8ed-4192-98f6-f7fc4d5c8100 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.493392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe0db1-56b4-473e-a06b-4446dc7a933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.494274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0db1-56b4-473e-a06b-4446dc7a933c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.494809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0db1-56b4-473e-a06b-4446dc7a933c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.494809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0db1-56b4-473e-a06b-4446dc7a933c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.509311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe0db1-56b4-473e-a06b-4446dc7a933c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.511210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe0db1-56b4-473e-a06b-4446dc7a933c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e1fe0db1-56b4-473e-a06b-4446dc7a933c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:18.532628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:18.533604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:18.534778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:18.534778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:18.534778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:18.534778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:18.534778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 354/729] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:18.535267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207300-f12f-4b3b-8ece-4ba2ad37613e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.535267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207300-f12f-4b3b-8ece-4ba2ad37613e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.535556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5207300-f12f-4b3b-8ece-4ba2ad37613e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.551372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.552720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e5207300-f12f-4b3b-8ece-4ba2ad37613e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:18.553898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e5207300-f12f-4b3b-8ece-4ba2ad37613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:18.567165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f0301-c8ed-4192-98f6-f7fc4d5c8100 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.604849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea701232-0782-4f72-993c-e3e6fa9b5ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.605630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701232-0782-4f72-993c-e3e6fa9b5ced None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.605914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701232-0782-4f72-993c-e3e6fa9b5ced None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.606309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea701232-0782-4f72-993c-e3e6fa9b5ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.617716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea701232-0782-4f72-993c-e3e6fa9b5ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.619277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea701232-0782-4f72-993c-e3e6fa9b5ced None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.631149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 355/730] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:18.633188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241d0c4-2d95-4d34-8f2d-2c4fd4ed9452 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.633725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241d0c4-2d95-4d34-8f2d-2c4fd4ed9452 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.634139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241d0c4-2d95-4d34-8f2d-2c4fd4ed9452 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ea701232-0782-4f72-993c-e3e6fa9b5ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:18.639417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:18.640524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:18.641564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:18.641564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:18.641564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:18.641564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:18.642138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 356/731] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:18.643032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241d0c4-2d95-4d34-8f2d-2c4fd4ed9452 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.644738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 376/732] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:18.646033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cbe4-b7f8-470c-82a9-78c71ecfebb5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.646323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cbe4-b7f8-470c-82a9-78c71ecfebb5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.646601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0cbe4-b7f8-470c-82a9-78c71ecfebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.664044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1fd0cbe4-b7f8-470c-82a9-78c71ecfebb5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:18.665217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1fd0cbe4-b7f8-470c-82a9-78c71ecfebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:18.683303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.736443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 377/733] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:18.738010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cbe4-b7f8-470c-82a9-78c71ecfebb5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.766918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241d0c4-2d95-4d34-8f2d-2c4fd4ed9452 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.802412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.803081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.803394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.803670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.804955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:18.805266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.806732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.813085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.818398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 357/734] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:18.819466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.847797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0cbe4-b7f8-470c-82a9-78c71ecfebb5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.886934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.887660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.888457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.888852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.889577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:18.889889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.891305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:18.910366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 378/735] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:18.911271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:18.930257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:18.937333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379fdcf-f2dd-4330-9b84-9ddff427542c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:18.976656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:18.977652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:18.977948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:18.978506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:18.979270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:18.981386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.020493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ea278-4b48-4c39-a3e8-ad5ef39e7e32 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.046862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.055052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.056307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.056744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.057032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.067404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:19.075972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:19.076969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:19.077888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 379/736] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:19.078578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.086428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 358/737] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:18 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:19.087546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.232730 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116549b3-ff4c-424e-9af9-2555b042c3e7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.254746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05d64a-2b7b-4707-893c-1e629bf548b5 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.271593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.272420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.272713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.272980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.282123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.283609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.294538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90e3ef-5958-4bf8-bb28-154300c5a2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.295373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90e3ef-5958-4bf8-bb28-154300c5a2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.295665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90e3ef-5958-4bf8-bb28-154300c5a2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.295938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90e3ef-5958-4bf8-bb28-154300c5a2b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:19.300512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:19.301449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 380/738] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:19.302801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.305474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90e3ef-5958-4bf8-bb28-154300c5a2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.307247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90e3ef-5958-4bf8-bb28-154300c5a2b6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-fe90e3ef-5958-4bf8-bb28-154300c5a2b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:19.323282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:19.324413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:19.326391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 359/739] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:19.328084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5255e-3aff-46a2-9da0-17abf77c01c6 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.328084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5255e-3aff-46a2-9da0-17abf77c01c6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.328084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5255e-3aff-46a2-9da0-17abf77c01c6 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.329453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 360/740] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:19.330221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5255e-3aff-46a2-9da0-17abf77c01c6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.346350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.358374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.410927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376a98f-0fad-4341-8726-a96555b2c908 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.439191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5255e-3aff-46a2-9da0-17abf77c01c6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.447570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.448440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.448885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.449167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.449921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.450247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.451647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.460331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 381/741] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:19.461232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.482307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5404f-7691-412a-932f-9b16ac75dfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.483036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5404f-7691-412a-932f-9b16ac75dfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.483311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5404f-7691-412a-932f-9b16ac75dfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.483594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5404f-7691-412a-932f-9b16ac75dfd1 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.484606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5404f-7691-412a-932f-9b16ac75dfd1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.484775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5404f-7691-412a-932f-9b16ac75dfd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.486183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5404f-7691-412a-932f-9b16ac75dfd1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.508733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 361/742] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:19.508733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07105df4-5d0b-4138-8304-9b49eea91abd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.508733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07105df4-5d0b-4138-8304-9b49eea91abd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.508733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07105df4-5d0b-4138-8304-9b49eea91abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.524945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-07105df4-5d0b-4138-8304-9b49eea91abd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:19.526206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-07105df4-5d0b-4138-8304-9b49eea91abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:19.571009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53330b52-299a-4925-892d-25ec57dca4fd None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.581312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.596200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.605664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 362/743] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 10:50:19.605664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07105df4-5d0b-4138-8304-9b49eea91abd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.622429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.623106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.623414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.623689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.624536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:19.624846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.626287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.645125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 382/744] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:19.650265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.708566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07105df4-5d0b-4138-8304-9b49eea91abd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.744847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.745579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.745865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.746150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.756354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.757950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.758611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12731d84-ec21-4738-b2dc-c42bc6816ea0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:19.796762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:19.797785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:19.798249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:19.798704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:19.837609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:19.875012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:19.876698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:19.890505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:19.891516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:19.892430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:19.892430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:19.892430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:19.892430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:19.892430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 363/745] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:19.892716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:19.954436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.003902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d9ff3-e1b1-4543-9f59-07d6fbf09da7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.005495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 383/746] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 10:50:20.006688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.043004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.043727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.044596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.044596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.044986 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.046355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.075616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.107656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7c6b0-99af-4a67-b871-2823d30d10d5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.122273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 364/747] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:19 2022] POST /identity/v3/services => generated 240 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:20.132237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.160760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.161455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.161748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.162027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.278205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.280384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.286859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd4d50-52e2-4173-b151-f01403367e5c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.302252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.328058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.328877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.329264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.329639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.330479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:20.330844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.332445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.349047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 365/748] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:20.355276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.363885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.413175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 384/749] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:20.414867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.466400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004cd0-cdf3-4e69-aadf-8a41e55833d4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.506109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.507001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.507405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.508365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.521879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.525281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.526548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39a21-26a5-4d93-907d-291cef26d300 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:20.543998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:20.544925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:20.545821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:20.545821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:20.545821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:20.545821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:20.545821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 366/750] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:20.546828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.566971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.568277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.568802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.569283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.575890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.578980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.580663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:20.596388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:20.597654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:20.598600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:20.601341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 385/751] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:20.602323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.630599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.661212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c76ccf-28ce-4f85-a647-48d639dc949c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.700976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.701844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.702205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.702562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.713078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee291e-6e31-46c4-a486-d219234c918e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.714098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.715703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:20.735382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:20.736319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:20.737219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:20.737219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:20.737219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:20.737219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:20.737219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 367/752] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:20.738078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.760208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.760888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.761181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.761464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.762528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:20.762858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.764290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.780291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 386/753] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:20.782026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.832770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.859093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf611549-fa8c-469a-8326-55e43089b7ab None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.891969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:20.905572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.906568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.907020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.907516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.908544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:20.909020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.909524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319463d4-413e-4c7b-8846-a5e2f3ffc617 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:20.911114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.934436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 368/754] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:20.935690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:20.950562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:20.951251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:20.951548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:20.951824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:20.952574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:20.952887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:20.954472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:20.972382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 387/755] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:20.973370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.051282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ff989-d774-42c5-8f7f-215c31d0a075 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.081714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73256-5b37-45ee-8f4e-97a34f172779 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.090429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.091634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.092137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.092711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.094794 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.101992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.104166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.119302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.120222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.120517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.120799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:21.121854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:21.122841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:21.123802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:21.126320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 369/756] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:21.127415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.133954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.135472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.138305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:21.154923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:21.156079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:21.156980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:21.156980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:21.156980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:21.156980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:21.156980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 388/757] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:21.157659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.244469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a0947-0240-4d86-9566-599cd2ec0d82 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.271387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b55822-4fac-4890-b538-4ae04efd9138 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.291438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.292551 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.293037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.293515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.300905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.302563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.315153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b7138-903a-46cf-b61f-64360fca6d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.315841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b7138-903a-46cf-b61f-64360fca6d24 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.316130 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b7138-903a-46cf-b61f-64360fca6d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.316411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b7138-903a-46cf-b61f-64360fca6d24 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.317204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b7138-903a-46cf-b61f-64360fca6d24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.317519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b7138-903a-46cf-b61f-64360fca6d24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.318943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b7138-903a-46cf-b61f-64360fca6d24 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:21.320821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:21.321975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:21.323053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:21.324893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 370/758] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:21.325838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.335341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 389/759] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:21.336919 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed17b-95cb-4a43-9656-fc179df72215 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.337200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed17b-95cb-4a43-9656-fc179df72215 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.337483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed17b-95cb-4a43-9656-fc179df72215 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.339076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 390/760] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:21.343534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ed17b-95cb-4a43-9656-fc179df72215 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.394275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.404520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.442724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c731e2-7603-4213-96fa-b9085d7e01e4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.449271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ed17b-95cb-4a43-9656-fc179df72215 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.480034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.480645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.480936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.481219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.481945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.482265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.483683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.487072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d1581-bc5b-4743-863a-00b31bcaea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.487764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d1581-bc5b-4743-863a-00b31bcaea59 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.488052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d1581-bc5b-4743-863a-00b31bcaea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.488332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d1581-bc5b-4743-863a-00b31bcaea59 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.489120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d1581-bc5b-4743-863a-00b31bcaea59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.489435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d1581-bc5b-4743-863a-00b31bcaea59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.490854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d1581-bc5b-4743-863a-00b31bcaea59 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.500206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 371/761] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:21.500206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.507260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 391/762] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:21.508943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadd9c8-6a72-4afa-ad11-ba3fe5a0651c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.509237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadd9c8-6a72-4afa-ad11-ba3fe5a0651c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.509518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadd9c8-6a72-4afa-ad11-ba3fe5a0651c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.526287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5eadd9c8-6a72-4afa-ad11-ba3fe5a0651c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:21.527514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5eadd9c8-6a72-4afa-ad11-ba3fe5a0651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:21.600838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 392/763] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:21.612018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadd9c8-6a72-4afa-ad11-ba3fe5a0651c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.619942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621bef6-91ab-41d7-84c2-0aec1c231582 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.636321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.657036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.657700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.657991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.658280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.659187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=fef3e61b2a6742ddba146be546996833) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.659523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.660951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.665931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:21.675398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 372/764] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/role_assignments?user.id=fef3e61b2a6742ddba146be546996833&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:21.677415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.731005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadd9c8-6a72-4afa-ad11-ba3fe5a0651c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.771583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.772447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.772735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.773705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.782982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.785596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:21.805153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:21.806391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:21.807472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:21.811336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 393/765] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:21.812300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:21.831948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa2de5-1221-4e5b-ab72-26709bec071f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:21.876700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:21.877679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:21.877968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:21.878256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:21.879206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=f766fee8481a4eccbc56d4af0c2afbf2) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:21.879540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:21.881844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:21.899669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 373/766] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/role_assignments?user.id=f766fee8481a4eccbc56d4af0c2afbf2&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:21.900775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.012523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15d87-391c-436f-80e5-f07c0ca49123 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.028514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.051659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.086036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.087317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.087781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.088219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.090029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.091660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:22.107939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:22.109169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:22.110232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:22.119628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 394/767] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:22.120711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.129605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2fc98-444c-47e1-b7c8-d63b49176a3b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.174771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.175628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.175929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.176215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.177322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.177646 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.179114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.197122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 374/768] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.203204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.264953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f20ec-563a-4e16-8e1d-8d6c06d24fe2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.327091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa6c1d-d822-420b-b09f-7477d33261f6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.330123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.349336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.350410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.350715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.352956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.352956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.352956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.354388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.360124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 395/769] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.368131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.399261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6947c8-fa65-422b-b958-7409147d4fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.400099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6947c8-fa65-422b-b958-7409147d4fc5 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.400394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6947c8-fa65-422b-b958-7409147d4fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.400673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6947c8-fa65-422b-b958-7409147d4fc5 None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/54bb42878fff477099bbf56a46e9eec0/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.404256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6947c8-fa65-422b-b958-7409147d4fc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=54bb42878fff477099bbf56a46e9eec0, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.406954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6947c8-fa65-422b-b958-7409147d4fc5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.425872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.481488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 375/770] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:22 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/54bb42878fff477099bbf56a46e9eec0/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:22.483080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e53be-1869-4343-8431-36f6eb73a9f5 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.483604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e53be-1869-4343-8431-36f6eb73a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.484053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e53be-1869-4343-8431-36f6eb73a9f5 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.490617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 376/771] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:22.492472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a740a2-ae3e-4903-afab-6976563160c6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.493213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a740a2-ae3e-4903-afab-6976563160c6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.495313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a740a2-ae3e-4903-afab-6976563160c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.521804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-95a740a2-ae3e-4903-afab-6976563160c6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:22.523110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-95a740a2-ae3e-4903-afab-6976563160c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:22.529082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b081cc-9dcc-4883-9af8-f43502218c02 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.609449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.610565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.611062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.611621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.612562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.613082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.615714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.624350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 396/772] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.631917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.666231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 377/773] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:22.698124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a740a2-ae3e-4903-afab-6976563160c6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.772813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.777414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a4431-b74b-4204-b56f-c5ab4435deac None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.821202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.821872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.822226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.822510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.823643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] RBAC: Adding query filter params (interface=public, service_id=4812e784f40d4265930e14e11d476577, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:22.823958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.825982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.835125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 397/774] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/endpoints?service_id=4812e784f40d4265930e14e11d476577&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:22.844214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:22.853350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:22.872255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a740a2-ae3e-4903-afab-6976563160c6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:22.954085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:22.954925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:22.955238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:22.955552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] PATH_INFO: `/v3/services/network` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:22.956250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:22.957642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:22.985067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:22.986358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:22.989160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:22.994848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 378/775] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services/network => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:22.999643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.089876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd232f-5376-48f8-80a3-2e122ce2095b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.144215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.144945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.145232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.145614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.146447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.148568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.162353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 398/776] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services => generated 772 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:23.162915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0f802-8829-4152-bbfd-209e7e2bb847 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.213334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.213951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.214239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.214518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.215286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] RBAC: Adding query filter params (name=network) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:23.215600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.217098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.224659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 379/777] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:23.232711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.284373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:23.412675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd690-6fea-4fce-9735-977f070e6df0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.467343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.467970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.468263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.468548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.469286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] RBAC: Adding query filter params (type=network) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:23.469601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.471003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.477658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 399/778] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:23.483615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.514653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:23.638517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55d79a-daa7-447e-9853-7b658b6c591c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.690177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc25662-8d2c-4da8-b17e-78d88808b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.690918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc25662-8d2c-4da8-b17e-78d88808b803 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.691210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc25662-8d2c-4da8-b17e-78d88808b803 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.691428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc25662-8d2c-4da8-b17e-78d88808b803 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.692236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc25662-8d2c-4da8-b17e-78d88808b803 None admin] RBAC: Adding query filter params (interface=public, service_id=da0214131cc648dbb31aed2e49fa75fa, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:23.692561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc25662-8d2c-4da8-b17e-78d88808b803 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.693934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc25662-8d2c-4da8-b17e-78d88808b803 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.706501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 380/779] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:23 2022] GET /identity/v3/endpoints?service_id=da0214131cc648dbb31aed2e49fa75fa&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:23.707584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:23.845801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:23.856824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3de85-f6a3-4013-90e7-72dd19b29666 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:23.910777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:23.911465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.911758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.912043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.916059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:23.916059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:23.925443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0e4d7-5e72-49b7-9d8c-2d31fb7f4bdb None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.925739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0e4d7-5e72-49b7-9d8c-2d31fb7f4bdb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.926025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0e4d7-5e72-49b7-9d8c-2d31fb7f4bdb None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.927423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 381/780] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:23.928127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 400/781] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:23 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:23.933193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951420e-83f2-41db-91ea-9f28bd85ef02 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:23.933494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951420e-83f2-41db-91ea-9f28bd85ef02 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:23.933785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951420e-83f2-41db-91ea-9f28bd85ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:23.977086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4951420e-83f2-41db-91ea-9f28bd85ef02 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:23.978361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4951420e-83f2-41db-91ea-9f28bd85ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:24.092240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 382/782] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:24.117805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.229150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3dfb8-751e-40b6-ba11-1d28ec60825d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.265349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.266392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.266823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.267271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.290217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:24.298248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:24.299397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:24.300455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:24.302826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 401/783] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:24.309635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951420e-83f2-41db-91ea-9f28bd85ef02 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.334117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.461478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951420e-83f2-41db-91ea-9f28bd85ef02 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.508618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0816a0-fbb0-4c56-a5fc-98620f584d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.509328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0816a0-fbb0-4c56-a5fc-98620f584d31 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.509615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0816a0-fbb0-4c56-a5fc-98620f584d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.509894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0816a0-fbb0-4c56-a5fc-98620f584d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.510658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0816a0-fbb0-4c56-a5fc-98620f584d31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:24.510969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0816a0-fbb0-4c56-a5fc-98620f584d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.512381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0816a0-fbb0-4c56-a5fc-98620f584d31 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.527350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 383/784] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:24.532923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.563027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.671921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59ca29-24a0-4dd5-a843-5ca503b13753 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.678298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4fb81-d5a7-4975-8080-0b1239846f81 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.678587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4fb81-d5a7-4975-8080-0b1239846f81 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.678866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4fb81-d5a7-4975-8080-0b1239846f81 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.680687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 384/785] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:24.689390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dfd1f-282f-457a-8f1d-8948348673bc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.689390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dfd1f-282f-457a-8f1d-8948348673bc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.689390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dfd1f-282f-457a-8f1d-8948348673bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.708673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-625dfd1f-282f-457a-8f1d-8948348673bc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:24.709867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-625dfd1f-282f-457a-8f1d-8948348673bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:24.738918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.739686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.739979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.740264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.753144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.754761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:24.779198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:24.780134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:24.781035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:24.781035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:24.781035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:24.781035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:24.781035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 402/786] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:24.783003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.864393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.876864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 385/787] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:24.901394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63cd0d-eb01-422c-82e3-ab13ad0cd61d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:24.920896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dfd1f-282f-457a-8f1d-8948348673bc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:24.939709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:24.940421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:24.941329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:24.941329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:24.941638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:24.945763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:24.945763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:24.945763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:24.958816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 403/788] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:24.966706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.130708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dfd1f-282f-457a-8f1d-8948348673bc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.132765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cd572-6db6-4df1-9813-fa85d77765c8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.167895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b85636-1930-4cf5-8221-306d17689fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.168735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b85636-1930-4cf5-8221-306d17689fbd None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.169047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b85636-1930-4cf5-8221-306d17689fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.169291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b85636-1930-4cf5-8221-306d17689fbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.170838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.179829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b85636-1930-4cf5-8221-306d17689fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:25.181934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e512e4e-d4b1-4999-a373-b850341b324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.183467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512e4e-d4b1-4999-a373-b850341b324f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.183945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512e4e-d4b1-4999-a373-b850341b324f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.195311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512e4e-d4b1-4999-a373-b850341b324f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-14b85636-1930-4cf5-8221-306d17689fbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:25.195445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:25.196417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 386/789] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:25.197273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00bb31-643f-44ff-b5f9-b0f5631804ab None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.198035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00bb31-643f-44ff-b5f9-b0f5631804ab None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.198035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00bb31-643f-44ff-b5f9-b0f5631804ab None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.198035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 387/790] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:25.199714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00bb31-643f-44ff-b5f9-b0f5631804ab None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.200577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e512e4e-d4b1-4999-a373-b850341b324f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.202799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e512e4e-d4b1-4999-a373-b850341b324f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2e512e4e-d4b1-4999-a373-b850341b324f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:25.224429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:25.225425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:25.226623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:25.231388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 404/791] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:25.233021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c0e4-f53d-465b-9427-fc5735665bf9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.233376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c0e4-f53d-465b-9427-fc5735665bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.233721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c0e4-f53d-465b-9427-fc5735665bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.254315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d494c0e4-f53d-465b-9427-fc5735665bf9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:25.255776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d494c0e4-f53d-465b-9427-fc5735665bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:25.289586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.327462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 405/792] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:25.328774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494c0e4-f53d-465b-9427-fc5735665bf9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.354602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00bb31-643f-44ff-b5f9-b0f5631804ab None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.395486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.396616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.397176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.397663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.399107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:25.399664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.401365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.411425 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 388/793] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:25.412503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.463925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494c0e4-f53d-465b-9427-fc5735665bf9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.482135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.501657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.504460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.504460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.504460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.504460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:25.504460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.505342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.529058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 406/794] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:25.529962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.532483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab80ef4-5b77-4693-a607-ad29d9b175c6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.562376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.572235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.575095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.575095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.575095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.586939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:25.595103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:25.596213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:25.597337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:25.600031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 389/795] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:25.601647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.643992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd0425-9e88-48a2-8bb8-6ad19b326385 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.681160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.681898 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.682260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.682540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.693303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.695838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:25.714174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:25.715092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:25.716056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:25.716056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:25.716056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:25.716056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:25.718893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff3282-730b-4a26-9e5a-ea6d845d23f6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.719972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 407/796] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:25.720816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.751281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.760236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.765935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.765935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.765935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.774172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.775922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:25.796602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:25.797589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:25.798493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:25.798493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:25.798493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:25.798493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:25.800502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 390/797] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:25.801500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.835159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969c3c1-ff27-435e-b28e-cdb8c099803f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.853664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:25.874810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.876151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.876584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.877008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.877912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:25.878377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.879996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.891238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 408/798] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:25.892369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:25.929023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57fe29-fd1c-42d1-8aea-393a1d69b4af None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:25.968234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d3476-9356-4718-845f-5f7581c53451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:25.968832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3476-9356-4718-845f-5f7581c53451 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.969118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3476-9356-4718-845f-5f7581c53451 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.969396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d3476-9356-4718-845f-5f7581c53451 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.970163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d3476-9356-4718-845f-5f7581c53451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:25.970423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d3476-9356-4718-845f-5f7581c53451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:25.972071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d3476-9356-4718-845f-5f7581c53451 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:25.984318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 391/799] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:25.985685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10eca4-9e4f-4978-ad0b-7ace2e3344b2 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:25.985976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10eca4-9e4f-4978-ad0b-7ace2e3344b2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:25.986253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10eca4-9e4f-4978-ad0b-7ace2e3344b2 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:25.991575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 392/800] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:25.992263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10eca4-9e4f-4978-ad0b-7ace2e3344b2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.001221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.014208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72a3ae-40c5-4c0c-a67b-ddf015b1ed57 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.052415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.053088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.053377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.053657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.054640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.054842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.056293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.072923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 409/801] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:26.072923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.107133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.111237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10eca4-9e4f-4978-ad0b-7ace2e3344b2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.147484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb8306-9204-43c0-aeca-88689c72068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.149354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb8306-9204-43c0-aeca-88689c72068f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.149665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb8306-9204-43c0-aeca-88689c72068f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.149947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb8306-9204-43c0-aeca-88689c72068f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.160920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb8306-9204-43c0-aeca-88689c72068f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.162479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb8306-9204-43c0-aeca-88689c72068f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d3cb8306-9204-43c0-aeca-88689c72068f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:26.180306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.181209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.182135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:26.182135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:26.182135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:26.182135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:26.183814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 393/802] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.185310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc6fda-e57d-441a-9774-d5b3c10f6e2e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.185797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc6fda-e57d-441a-9774-d5b3c10f6e2e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.186093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca2ba8-6b0c-4b7c-ad52-16dd82a94ef3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.186451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc6fda-e57d-441a-9774-d5b3c10f6e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.206111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a4bc6fda-e57d-441a-9774-d5b3c10f6e2e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:26.207324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a4bc6fda-e57d-441a-9774-d5b3c10f6e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:26.230446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.231356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.231758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.232043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.239664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.241135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:26.256462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.257507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:26.258455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 410/803] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.259088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.272580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.290439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 394/804] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:26.291882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc6fda-e57d-441a-9774-d5b3c10f6e2e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.374571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7389a0d-f018-40f1-bc47-36e8e2da12b9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.375500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.395519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc6fda-e57d-441a-9774-d5b3c10f6e2e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.408293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.409088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.409403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.409689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.417228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.418674 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.429729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.430388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.430674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.430958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.431707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.432016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:26.432964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.433966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:26.434848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:26.435304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.435375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 411/805] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.435375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.449691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 395/806] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:26.450504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.511279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.543506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dff093-133e-4acc-9d68-77acc001e485 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.550628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b8f6-2ece-4f7d-b7b3-2c1c9110089e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.578303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35169-22f1-412b-b693-54ffadb60ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.578990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35169-22f1-412b-b693-54ffadb60ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.579310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35169-22f1-412b-b693-54ffadb60ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.579586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35169-22f1-412b-b693-54ffadb60ff4 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.580418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35169-22f1-412b-b693-54ffadb60ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.580728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35169-22f1-412b-b693-54ffadb60ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.582145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35169-22f1-412b-b693-54ffadb60ff4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.585028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.585452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.585798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.586021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.586677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.588058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.599285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 412/807] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.600192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.601379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 10:50:26.602543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:26.604027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ecd48-3a59-4c78-beb7-9286e6ad6f03 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.604027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ecd48-3a59-4c78-beb7-9286e6ad6f03 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.604027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ecd48-3a59-4c78-beb7-9286e6ad6f03 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.604299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 396/808] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.604299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.604423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 413/809] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:26.604423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ecd48-3a59-4c78-beb7-9286e6ad6f03 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.615321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.707558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f9b8f-1ad2-4ee4-b4fb-a667ebd6c629 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.709466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ecd48-3a59-4c78-beb7-9286e6ad6f03 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.743122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39200d01-6985-40f6-a370-71b3f4cf8a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.743823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39200d01-6985-40f6-a370-71b3f4cf8a37 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.744107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39200d01-6985-40f6-a370-71b3f4cf8a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.744380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39200d01-6985-40f6-a370-71b3f4cf8a37 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.745184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39200d01-6985-40f6-a370-71b3f4cf8a37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.745499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39200d01-6985-40f6-a370-71b3f4cf8a37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.746964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39200d01-6985-40f6-a370-71b3f4cf8a37 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.760802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 397/810] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:26.762181 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0708857-ac76-419a-a9b4-206fa6cc0f57 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.762417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0708857-ac76-419a-a9b4-206fa6cc0f57 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.762694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0708857-ac76-419a-a9b4-206fa6cc0f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.780503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e0708857-ac76-419a-a9b4-206fa6cc0f57 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:26.781724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e0708857-ac76-419a-a9b4-206fa6cc0f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:26.844837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.851004 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:26.852724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 398/811] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:26.855065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0708857-ac76-419a-a9b4-206fa6cc0f57 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.859982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.860779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.861067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.861349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.868809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:26.870256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:26.885849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:26.887376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 414/812] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:26.888361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:26.961878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0708857-ac76-419a-a9b4-206fa6cc0f57 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.989491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f17e42-2d8d-426e-a5fe-74836a8de37c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:26.996152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:26.997211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:26.997525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:26.997805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:26.998514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:26.998820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.000223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.004677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 399/813] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.005420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.023889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.024508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.024792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.025065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.035388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.036837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:27.053460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:27.055074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:27.056016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:27.056016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:27.056016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:27.056016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:27.056016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 415/814] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:27.056305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.070050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.094625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.111417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338fa0b-7c92-443e-ad3b-597755010c7d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.144405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.145298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.145586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.145868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/fef3e61b2a6742ddba146be546996833/roles/4fc9795c5001472c86ae32739f03b3f6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.153888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3a2b-267c-49b7-9c53-2b56da618f73 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.160281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=fef3e61b2a6742ddba146be546996833, role_id=4fc9795c5001472c86ae32739f03b3f6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.161727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.187093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.187927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.188202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.188482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] PATH_INFO: `/v3/services/network` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.189139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.190492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:27.202760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:27.205030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:27.205873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 416/815] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services/network => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:27.206660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.228116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 400/816] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:27 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/fef3e61b2a6742ddba146be546996833/roles/4fc9795c5001472c86ae32739f03b3f6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:27.229263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.332514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6753bac-57a7-4f23-bd29-115a1053915d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.334332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.368091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a9078-56ff-4d82-9777-fbacc127c59b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.385337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.396307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.399261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.399261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.399261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.399261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.399261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.401766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.420865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.421635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.422052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.422483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.430268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.430268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.430268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.430505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 417/817] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.430505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.431402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 401/818] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.432362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.548134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c201-721e-4975-99cf-73fe39394857 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.552287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852f4c4-a4b1-437e-ba59-9b43aee88130 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.586709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.587862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.588364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.588702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.589508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.589879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.591343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.597105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.597778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.598063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.598344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.599046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] RBAC: Adding query filter params (name=network) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:27.604863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.605139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.608480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 418/819] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.609391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.610762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 402/820] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:27.611657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.640644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.662720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.731058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404d96b-79ee-460d-b3af-b1eeaebeb975 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.736060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2668-604d-45c3-8eb1-4def6fdf9a96 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:27.767310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.767973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.768261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.768547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.769290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.770662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.774003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:27.774689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:27.774986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:27.775292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:27.842006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:27.843408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:27.864010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:27.895290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 403/821] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:27.896978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:27.967473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 419/822] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:27.968508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.048984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.072328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbcfa7-b34a-42c3-b740-c920fb142d75 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.090024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23b6e9-a1b0-4800-a89f-e680314c65f8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.120400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.122921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.123774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.124066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.124346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.131759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.133354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.136069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.136197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.136388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.136750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.137479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] RBAC: Adding query filter params (type=network) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:28.138264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.139137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.145204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 420/823] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:28.145744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:28.150408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:28.151379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:28.152296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 404/824] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:28.152781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.260368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a99da7-c24a-4e02-bc9a-3a52f42dc0e9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.281354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec1572-16dc-4179-b533-6310731a6712 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.299714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.300528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.300820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.301101 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.302175 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.303584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.321074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.324435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.324435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.324435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.339544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.339544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:28.354656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:28.355821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:28.356959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:28.356959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:28.356959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:28.356959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:28.358335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 421/825] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:28.359431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.409140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.474597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e0a7d-eb9a-4eb3-a164-29bd489dfd7a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.510455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.511490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.511919 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.512351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.513294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:28.513757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.515443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.521770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 405/826] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:28.524962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.549499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 422/827] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:28.552352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.556449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.744281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.750484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc227-17b1-4c27-b279-bebe27b62399 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.752716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9c182-4057-4f32-ae46-4fb1a33f01cd None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.787000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.787983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.788894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.789173 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/f766fee8481a4eccbc56d4af0c2afbf2/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.791229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=f766fee8481a4eccbc56d4af0c2afbf2, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.792675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.794263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.794992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.795322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.795607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.796344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:28.796656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.798057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:28.810903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 406/828] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:28.816015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:28.885683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 423/829] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:28 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/f766fee8481a4eccbc56d4af0c2afbf2/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:28.888330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:28.941073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df194b-7d48-4394-acb1-c4ba65cff7eb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:28.985415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:28.986707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:28.987246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:28.987741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:28.996428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:28.998261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:29.015110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:29.016176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:29.017062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:29.019334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 407/830] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:29.026354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:29.138183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c232-d5f0-4e3f-9417-f3208014d541 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:29.175026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f71f8-cfd2-4224-ac1f-8fecb95d43d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:29.176141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f71f8-cfd2-4224-ac1f-8fecb95d43d3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.176630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f71f8-cfd2-4224-ac1f-8fecb95d43d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.177111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f71f8-cfd2-4224-ac1f-8fecb95d43d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.178160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f71f8-cfd2-4224-ac1f-8fecb95d43d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:29.178734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f71f8-cfd2-4224-ac1f-8fecb95d43d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:29.180507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f71f8-cfd2-4224-ac1f-8fecb95d43d3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:29.193662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 424/831] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:29.200317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:29.313038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0be337-686d-472c-b633-560052ea64a3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:29.338310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:29.351528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81939651-ada3-4daf-8bef-e8519a20fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:29.352474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81939651-ada3-4daf-8bef-e8519a20fa34 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.352969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81939651-ada3-4daf-8bef-e8519a20fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.358929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81939651-ada3-4daf-8bef-e8519a20fa34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.358929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81939651-ada3-4daf-8bef-e8519a20fa34 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=54bb42878fff477099bbf56a46e9eec0) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:29.358929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81939651-ada3-4daf-8bef-e8519a20fa34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:29.358929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81939651-ada3-4daf-8bef-e8519a20fa34 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:29.363204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 408/832] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:29 2022] GET /identity/v3/role_assignments?user.id=54bb42878fff477099bbf56a46e9eec0&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:29.402713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85e331-620c-4645-804f-c2ae0c15bfd3 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.403667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85e331-620c-4645-804f-c2ae0c15bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.404281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85e331-620c-4645-804f-c2ae0c15bfd3 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.406193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 425/833] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:29.412145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb050de-c98e-40a6-9afa-488eb63be972 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.412776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb050de-c98e-40a6-9afa-488eb63be972 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.413490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb050de-c98e-40a6-9afa-488eb63be972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.433619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-afb050de-c98e-40a6-9afa-488eb63be972 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:29.435580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-afb050de-c98e-40a6-9afa-488eb63be972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:29.515903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 409/834] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:29.540956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85e331-620c-4645-804f-c2ae0c15bfd3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:29.562116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:29.650280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85e331-620c-4645-804f-c2ae0c15bfd3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:29.689606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:29.690752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.691289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.691819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.704560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:29.713682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:29.715015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:50:29.715983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:29.718709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 426/835] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:29.724591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb050de-c98e-40a6-9afa-488eb63be972 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:29.834635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb050de-c98e-40a6-9afa-488eb63be972 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:29.853691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:29.872964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:29.873976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:29.874472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:29.875009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:29.876024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:29.876544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:29.878134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:29.888728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 410/836] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:29.895252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:29.991303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.015921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779de3c-994c-4fed-8eab-3d8808ccf568 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.055516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744352a-edf6-420a-a6ec-a80fc25267d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.057106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744352a-edf6-420a-a6ec-a80fc25267d9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.057596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744352a-edf6-420a-a6ec-a80fc25267d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.058073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744352a-edf6-420a-a6ec-a80fc25267d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.069001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744352a-edf6-420a-a6ec-a80fc25267d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.071365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744352a-edf6-420a-a6ec-a80fc25267d9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7744352a-edf6-420a-a6ec-a80fc25267d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:30.091877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:30.092882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:30.093787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:30.093787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:30.093787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:30.093787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:30.095814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 427/837] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:30.102739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.118277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc25e2-ae3e-4bfb-974a-6eb76c9728bd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.118608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc25e2-ae3e-4bfb-974a-6eb76c9728bd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.118841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc25e2-ae3e-4bfb-974a-6eb76c9728bd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.120543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 428/838] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:30.126090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaebfa3-1568-46f7-b59f-f0ddc7353f17 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.126425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaebfa3-1568-46f7-b59f-f0ddc7353f17 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.126655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaebfa3-1568-46f7-b59f-f0ddc7353f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.144759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3aaebfa3-1568-46f7-b59f-f0ddc7353f17 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:30.145952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3aaebfa3-1568-46f7-b59f-f0ddc7353f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:30.227377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 429/839] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:30.235413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed7854-942a-4add-afd9-47428dd35901 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.242886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaebfa3-1568-46f7-b59f-f0ddc7353f17 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.289528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35243aac-c672-483c-8427-708bd820b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.290616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35243aac-c672-483c-8427-708bd820b926 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.291518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35243aac-c672-483c-8427-708bd820b926 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.292040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35243aac-c672-483c-8427-708bd820b926 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.293634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35243aac-c672-483c-8427-708bd820b926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.294178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35243aac-c672-483c-8427-708bd820b926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.296524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35243aac-c672-483c-8427-708bd820b926 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.308677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.321078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 411/840] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:30.326404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35243aac-c672-483c-8427-708bd820b926 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.361745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaebfa3-1568-46f7-b59f-f0ddc7353f17 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.375810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.397142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.398550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.398975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.399282 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.399978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.401341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:30.414356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:30.415465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:30.416433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:30.420478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 430/841] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:30.421732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.449393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35243aac-c672-483c-8427-708bd820b926 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.490029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.491144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.495319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.495825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.503871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.505579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:30.522648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:30.524282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:30.525231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:30.526981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63215eeb-2f29-4cc7-a8b0-2a316c2a2a83 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.533707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 412/842] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:30.536012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.553811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.566337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.567005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.570089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.570089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.570089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.570089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.570851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.580147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 431/843] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:30.581410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.649224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.654314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc61b28-76ad-461c-9edb-64c5ea4b9612 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.691874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50033a1e-beff-4804-bbb5-d5f1f3ce6a19 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.696974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.697918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.698404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.698880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.699937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.700438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.702079 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.727280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 413/844] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:30.729356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:30.732480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57745be4-3bd9-4469-8dd1-333aeb5c849c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.733151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57745be4-3bd9-4469-8dd1-333aeb5c849c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.733438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57745be4-3bd9-4469-8dd1-333aeb5c849c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.733717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57745be4-3bd9-4469-8dd1-333aeb5c849c None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.734431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57745be4-3bd9-4469-8dd1-333aeb5c849c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:30.734738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57745be4-3bd9-4469-8dd1-333aeb5c849c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.736265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57745be4-3bd9-4469-8dd1-333aeb5c849c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:30.749717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 432/845] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:30.857251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791f0d-dcd0-4d48-af1e-ad73c015938e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:30.902729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:30.903779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:30.904299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:30.904816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:30.933352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:30.981583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:30.983085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.083969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c43f-3ada-4f44-aac4-d441618cb1ba None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.085589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c43f-3ada-4f44-aac4-d441618cb1ba None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.086076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c43f-3ada-4f44-aac4-d441618cb1ba None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.087955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 433/846] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:31.091110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 414/847] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:50:31.093755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8d516-b2b6-41df-b7a2-bdf5e1ecfd07 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.094291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8d516-b2b6-41df-b7a2-bdf5e1ecfd07 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.094855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8d516-b2b6-41df-b7a2-bdf5e1ecfd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.095716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.120461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a2e8d516-b2b6-41df-b7a2-bdf5e1ecfd07 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:31.121736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a2e8d516-b2b6-41df-b7a2-bdf5e1ecfd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:31.156680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.173367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.203301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 434/848] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:31.204987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b39c4-3436-4b29-8944-995b0fdd1962 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.223454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8d516-b2b6-41df-b7a2-bdf5e1ecfd07 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.243732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d959da-829c-4965-809c-69d8c953556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.244631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d959da-829c-4965-809c-69d8c953556a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.245021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d959da-829c-4965-809c-69d8c953556a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.245371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d959da-829c-4965-809c-69d8c953556a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.258236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d959da-829c-4965-809c-69d8c953556a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.259885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d959da-829c-4965-809c-69d8c953556a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-52d959da-829c-4965-809c-69d8c953556a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:31.278922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:31.279894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:31.280803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:31.280803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:31.280803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:31.280803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:31.280803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 415/849] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:31.286679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d959da-829c-4965-809c-69d8c953556a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.345734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8d516-b2b6-41df-b7a2-bdf5e1ecfd07 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.389975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.391346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.391845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.392374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.405210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:31.405941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d959da-829c-4965-809c-69d8c953556a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:31.414412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:31.415453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:50:31.416314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:31.417772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.420114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 435/850] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:31.426398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.453481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.454187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.454482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.454764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.455548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:31.456303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.458002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.478428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 416/851] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:31.479619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.484513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.563754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be1f7-4d36-4fde-91ae-b3160bb56980 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.599195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd58e-a8ac-4e25-96ec-064462861cab None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.610899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b9d1a-a706-4bf7-8888-f5a8cbe404d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.612065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b9d1a-a706-4bf7-8888-f5a8cbe404d5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.612544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b9d1a-a706-4bf7-8888-f5a8cbe404d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.612829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b9d1a-a706-4bf7-8888-f5a8cbe404d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.613586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b9d1a-a706-4bf7-8888-f5a8cbe404d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:31.613910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b9d1a-a706-4bf7-8888-f5a8cbe404d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.615304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b9d1a-a706-4bf7-8888-f5a8cbe404d5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.627533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 436/852] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:31.627533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23ca2d-dac3-47ca-81a8-c27a3fcac146 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.627533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23ca2d-dac3-47ca-81a8-c27a3fcac146 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.627533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23ca2d-dac3-47ca-81a8-c27a3fcac146 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.628663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 437/853] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:31.634345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a591a95-5554-4943-b6de-9536f01278b9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.634666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a591a95-5554-4943-b6de-9536f01278b9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.634999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a591a95-5554-4943-b6de-9536f01278b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.640288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.641889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.642179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.642457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.650351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.652294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7a591a95-5554-4943-b6de-9536f01278b9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:31.652427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.653498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7a591a95-5554-4943-b6de-9536f01278b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:31.666368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:31.667570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:31.668504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 417/854] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:31.669026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.702948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.733123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 438/855] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:31.734047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a591a95-5554-4943-b6de-9536f01278b9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.766783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.790031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aad63-91d2-4bde-96e3-f7c1f8266b75 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.828130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.828917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.829201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.829476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.837619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a591a95-5554-4943-b6de-9536f01278b9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:31.840045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.841478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:31.861822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:31.863217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:31.864152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:31.864152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:31.864152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:31.864152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:31.864152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 418/856] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:31.864152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.879409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:31.880043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:31.880337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:31.880622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:31.881474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:31.882057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:31.883632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:31.901254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 439/857] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:31.901254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:31.958347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:31.978917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790216ae-65e0-4820-9bc1-0245b0bd216e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.007475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.014376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.015217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.015518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.018454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.018454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.018634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345de5-612f-4b38-b67a-ceae557d8306 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.018711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:32.031595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:32.032604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:32.034034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 419/858] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:32.034934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.058765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.059477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.059773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.060040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.060766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.061333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.062445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.078808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 440/859] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:32.079458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.146248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca10f49-178d-4a5b-af27-e6f8c1cfb7d2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.182192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.182851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.183138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.183449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.184279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=fef3e61b2a6742ddba146be546996833) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.184594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.186040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.186460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da033ec9-bc9b-4b82-84ae-ab3bc7cd05c3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.192054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 420/860] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/role_assignments?user.id=fef3e61b2a6742ddba146be546996833&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:32.192827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.216287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.226511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.227555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.227845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.228131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.228869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.229183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.230607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.236183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 441/861] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:32.242394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.268680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.334603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491842c-c4a5-4a78-9a0d-743d59bb7cf1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.375232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.376384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.376678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.377209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.386121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.386629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b293-1eef-403b-92fb-8818603d0b52 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.387600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:32.404729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:32.405903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:32.406865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 421/862] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:32.412057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.437876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.438560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.438851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.439136 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.439891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.440209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.441594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.449781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 442/863] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:32.499697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.544492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1710533-b252-4630-85a3-b4d6be80db27 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.585442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eae47b-47bb-4366-9cb3-901daf687968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.586795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae47b-47bb-4366-9cb3-901daf687968 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.587380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae47b-47bb-4366-9cb3-901daf687968 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.587885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae47b-47bb-4366-9cb3-901daf687968 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.588981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eae47b-47bb-4366-9cb3-901daf687968 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:32.589509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eae47b-47bb-4366-9cb3-901daf687968 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.591218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eae47b-47bb-4366-9cb3-901daf687968 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.613635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 422/864] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:32.619475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.710396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.733258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c685f3b-a74a-4868-87d7-02b4626434f4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:32.769304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:32.769962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.770249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.770530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.841419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:32.842879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:32.861511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de56e0-793a-444d-a4e1-c6453667c1a8 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.864925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de56e0-793a-444d-a4e1-c6453667c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.864925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de56e0-793a-444d-a4e1-c6453667c1a8 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.864925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 423/865] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:32.872382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5b29-845e-4392-99a1-6b7cc809e6da None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:32.873109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5b29-845e-4392-99a1-6b7cc809e6da None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:32.873615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f5b29-845e-4392-99a1-6b7cc809e6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:32.898342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-897f5b29-845e-4392-99a1-6b7cc809e6da None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:32.905353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-897f5b29-845e-4392-99a1-6b7cc809e6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:32.948840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:32.973811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 443/866] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:32.983622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:32.995338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 424/867] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:33.015888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f5b29-845e-4392-99a1-6b7cc809e6da None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.077215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:33.106000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19909-38b5-4652-aa70-1e4b87ce3b00 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.127759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f5b29-845e-4392-99a1-6b7cc809e6da None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.152757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.154579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.155023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.155744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.166938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.167752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.168400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.168519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.169345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.170693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.170904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.172492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:33.191120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:33.192625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:33.193534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:33.193534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:33.193534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:33.193534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:33.198443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 444/868] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:33.199156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.237670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:33.238804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 425/869] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:33 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:33.324168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd55f3c-a160-43a7-868c-97453b0c85f3 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.372246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.373347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.373835 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.374328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.375295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:33.375855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.377450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.390676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 445/870] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:33.396417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.506280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0427bbd-277c-40e5-9a9f-553d8d1d6646 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.526121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:33.546862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.548072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.548558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.549029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.556317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.558045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:33.573816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:33.574793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:33.575743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:33.578424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 426/871] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:33.584547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.687032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:33.700606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6bf1c-f4b7-4ef0-ae8d-dd3a5f133470 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.735644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6c7c5-02d8-4b88-9577-82b4994189f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.736673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c7c5-02d8-4b88-9577-82b4994189f8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.737168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c7c5-02d8-4b88-9577-82b4994189f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.737640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c7c5-02d8-4b88-9577-82b4994189f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.738642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6c7c5-02d8-4b88-9577-82b4994189f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:33.739151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6c7c5-02d8-4b88-9577-82b4994189f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.740856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6c7c5-02d8-4b88-9577-82b4994189f8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.754658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 446/872] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:33.761583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:33.863697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12336d2-2fc0-472e-a939-62fa0fbeb26b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:33.899543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fd5cf-34ab-4f54-949b-bea8a9315b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:33.901019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fd5cf-34ab-4f54-949b-bea8a9315b5e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:33.901516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fd5cf-34ab-4f54-949b-bea8a9315b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:33.901995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fd5cf-34ab-4f54-949b-bea8a9315b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:33.903026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7fd5cf-34ab-4f54-949b-bea8a9315b5e None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=f766fee8481a4eccbc56d4af0c2afbf2) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:33.903525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7fd5cf-34ab-4f54-949b-bea8a9315b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:33.905797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7fd5cf-34ab-4f54-949b-bea8a9315b5e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:33.912286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 427/873] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:33 2022] GET /identity/v3/role_assignments?user.id=f766fee8481a4eccbc56d4af0c2afbf2&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:34.486429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e1e77-1e7d-4ba1-bf29-9c6197fc5e39 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.487514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e1e77-1e7d-4ba1-bf29-9c6197fc5e39 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.488031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e1e77-1e7d-4ba1-bf29-9c6197fc5e39 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.489915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 447/874] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:34.496053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37d84d-369f-4576-a6bb-19b724fa4cac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.496644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37d84d-369f-4576-a6bb-19b724fa4cac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.497135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37d84d-369f-4576-a6bb-19b724fa4cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.514272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-aa37d84d-369f-4576-a6bb-19b724fa4cac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:34.515790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aa37d84d-369f-4576-a6bb-19b724fa4cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:34.529574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de8b3f-7a03-4aeb-80f9-4c6b0e341c99 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.530158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de8b3f-7a03-4aeb-80f9-4c6b0e341c99 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.530645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de8b3f-7a03-4aeb-80f9-4c6b0e341c99 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.532463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 448/875] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:34.537649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a449c-0155-43b9-8157-1d3a2fe8c1d2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.538389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a449c-0155-43b9-8157-1d3a2fe8c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.538877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a449c-0155-43b9-8157-1d3a2fe8c1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.554975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:34.559583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-382a449c-0155-43b9-8157-1d3a2fe8c1d2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:34.561061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-382a449c-0155-43b9-8157-1d3a2fe8c1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:34.593669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 428/876] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:34.621494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37d84d-369f-4576-a6bb-19b724fa4cac None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:34.647756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 449/877] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:34.670259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a449c-0155-43b9-8157-1d3a2fe8c1d2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:34.750270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37d84d-369f-4576-a6bb-19b724fa4cac None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:34.763573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:34.786265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a449c-0155-43b9-8157-1d3a2fe8c1d2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:34.788602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:34.789748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.790235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.790737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.801850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:34.811154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:34.812132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:34.812982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:34.820515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 429/878] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:34.822075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:34.823860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:34.824531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.824819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.825103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.825895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:34.828550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:34.857412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:34.932589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8a395-0bde-442c-aee5-1eace5f184cf None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:34.959480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 450/879] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:34 2022] POST /identity/v3/services => generated 269 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:34.983903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:34.984859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:34.985380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:34.985812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:34.986831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:34.987345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:34.989117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.006467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 430/880] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:35.007539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.124174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26638e70-74d7-4f50-bc1d-051667748f47 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.160595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a921-235e-4b7b-a09d-33a7a5120204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.161392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a921-235e-4b7b-a09d-33a7a5120204 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.161673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a921-235e-4b7b-a09d-33a7a5120204 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.161954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a921-235e-4b7b-a09d-33a7a5120204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.170261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.174089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7a921-235e-4b7b-a09d-33a7a5120204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.175588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7a921-235e-4b7b-a09d-33a7a5120204 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-00d7a921-235e-4b7b-a09d-33a7a5120204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:35.193547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:35.194612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:35.196653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:35.196653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:35.196653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:35.196653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:35.196653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 451/881] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:35.200782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.296803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.308906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6241a5-39a9-47e7-811f-a7487da6c33d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.347707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550fbe4-8b1a-4797-b724-d02f5bcb091b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.348032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.348235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550fbe4-8b1a-4797-b724-d02f5bcb091b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.348386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550fbe4-8b1a-4797-b724-d02f5bcb091b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.348579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.348873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.349157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.349897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:35.350015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 452/882] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:35.350216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.352320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.358873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b866-65de-4e2a-bc9c-a5b1391d195c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.359168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b866-65de-4e2a-bc9c-a5b1391d195c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.359475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4b866-65de-4e2a-bc9c-a5b1391d195c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.368450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 431/883] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:35.375216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.377111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-94b4b866-65de-4e2a-bc9c-a5b1391d195c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:35.378309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-94b4b866-65de-4e2a-bc9c-a5b1391d195c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:35.455426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 453/884] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:35.473695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4b866-65de-4e2a-bc9c-a5b1391d195c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.482935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a9c5-b80a-41b1-bc68-f50dd296a86a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.529291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.531090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.532123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.532629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.544449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.547095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:35.565958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:35.567000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:35.567994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 432/885] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:35.574183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.586366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.595733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4b866-65de-4e2a-bc9c-a5b1391d195c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.622110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.641146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.642517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.643012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.643650 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.644776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.646481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:35.660229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:35.661292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:35.662170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:35.670515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 454/886] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:35.671796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.688221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471561b-bb0e-4d28-81db-9cf4ab0275d4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.725861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.726463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.726816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.727023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.727901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:35.728216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.729923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.749049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 433/887] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:35.754786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.788865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd33a4-1f7b-4a5c-835b-3b55ed6bad83 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.828658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.829672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.830146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.830643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.831221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.832890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:35.833416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.835047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:35.845434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 455/888] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:35.846844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:35.869780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcad81-29a8-4921-bd8a-93122a494f15 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.904571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:35.904832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e798b13-30b8-4e04-a562-546a4d67de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:35.905598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798b13-30b8-4e04-a562-546a4d67de11 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:35.905769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798b13-30b8-4e04-a562-546a4d67de11 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:35.906118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e798b13-30b8-4e04-a562-546a4d67de11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:35.965658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ce76-2f54-459a-b508-61744ae43893 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:35.977415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e798b13-30b8-4e04-a562-546a4d67de11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:35.978874 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e798b13-30b8-4e04-a562-546a4d67de11 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.005746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.007264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.007765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.008165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.008987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:36.009405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.011090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.021870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 456/889] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:36.023565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.044997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.081343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 434/890] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:36.082614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a81d49-5274-4015-b799-1cee49c52a79 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.082906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a81d49-5274-4015-b799-1cee49c52a79 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.083198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a81d49-5274-4015-b799-1cee49c52a79 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.085113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 435/891] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:36.090251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb05f6-9c50-4359-8365-d489035ce182 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.090543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb05f6-9c50-4359-8365-d489035ce182 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.090828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb05f6-9c50-4359-8365-d489035ce182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.108154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2ccb05f6-9c50-4359-8365-d489035ce182 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:36.109348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2ccb05f6-9c50-4359-8365-d489035ce182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:36.136899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59f709-659b-419a-999d-881c52188f80 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.150921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.176077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.177230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.178010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.178815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.179978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] RBAC: Adding query filter params (interface=public, service_id=5220223587934138b0f77c02b1c2b77e, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:36.180612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.182191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 436/892] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:36.187457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb05f6-9c50-4359-8365-d489035ce182 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.187547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.190251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 457/893] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/endpoints?service_id=5220223587934138b0f77c02b1c2b77e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:36.196131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.294829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.301895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb05f6-9c50-4359-8365-d489035ce182 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.306588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439dc7d-56b6-4f71-9b3f-7703897f7275 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.339649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.340462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.340752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.341035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.341574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.342246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.342535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.342818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.343537 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.344900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.352206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.353650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.355784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 458/894] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:50:36.357437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:36.373527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:36.374525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:36.375481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:36.375481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:36.375481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:36.375481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:36.375481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 437/895] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:36.381935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.420959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.490539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61441c31-cd9d-432b-a48d-4a392eee1755 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.516081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52474ad-b0ec-4b8d-869e-5626cb3d2845 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.529758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.530699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.531082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.531480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.532239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.533698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:36.546393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:36.547630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:36.548666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 459/896] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:36.556093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.560977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.561850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.562294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.562727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.563661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:36.564123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.565654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.579508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 438/897] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:36.588160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.600035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.692844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1e329-9eec-4279-8b3f-eed9d39fcc77 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.693294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed271d-02be-4f91-96e3-caba53a20d0f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.697968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.732527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.733484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.733868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.734265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.742337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.743460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.744147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.744485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.745097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.745559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.746701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:36.747623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.749246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.754872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 460/898] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:36.763325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:36.763490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:36.764335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:36.765231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 439/899] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:36.772594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:36.869862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:36.880863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869f707-42de-4435-9607-c0d400ef32fe None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.921982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07d5cd-45e2-4f9f-a1c5-f4573e6887b2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:36.932961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.933766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.934176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.934558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.935431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:36.935854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.937469 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.942938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 461/900] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:36.973294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc5fd4-0d49-409f-85c9-578745fa7b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:36.974042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc5fd4-0d49-409f-85c9-578745fa7b4d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:36.974341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc5fd4-0d49-409f-85c9-578745fa7b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:36.974628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc5fd4-0d49-409f-85c9-578745fa7b4d None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:36.976715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc5fd4-0d49-409f-85c9-578745fa7b4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:36.977063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc5fd4-0d49-409f-85c9-578745fa7b4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:36.978623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc5fd4-0d49-409f-85c9-578745fa7b4d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:36.993765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 440/901] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:36.999975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:37.036598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:37.112764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157e61-2141-4594-8ef6-152798c9cb84 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:37.149540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a1966-b5bb-4bd7-a6bf-229a15c64420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:37.150126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1966-b5bb-4bd7-a6bf-229a15c64420 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.150725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1966-b5bb-4bd7-a6bf-229a15c64420 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.151005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1966-b5bb-4bd7-a6bf-229a15c64420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.151880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a1966-b5bb-4bd7-a6bf-229a15c64420 None admin] RBAC: Adding query filter params (role.id=3103c74eb4f5404dbb37aaba54d4aac4, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=fef3e61b2a6742ddba146be546996833) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:37.152199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a1966-b5bb-4bd7-a6bf-229a15c64420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:37.153640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a1966-b5bb-4bd7-a6bf-229a15c64420 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:37.159842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 462/902] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:36 2022] GET /identity/v3/role_assignments?user.id=fef3e61b2a6742ddba146be546996833&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=3103c74eb4f5404dbb37aaba54d4aac4 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:37.231105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2ac41-ed7e-4b9c-a43a-55475bf04165 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.231827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2ac41-ed7e-4b9c-a43a-55475bf04165 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.232337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2ac41-ed7e-4b9c-a43a-55475bf04165 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.234269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 441/903] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:37.240116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4c00-9752-421d-8b76-ea8289f469ac None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.240759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4c00-9752-421d-8b76-ea8289f469ac None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.241494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4c00-9752-421d-8b76-ea8289f469ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.260613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8f7c4c00-9752-421d-8b76-ea8289f469ac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:37.262234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8f7c4c00-9752-421d-8b76-ea8289f469ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:37.338902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 463/904] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:37.362924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2ac41-ed7e-4b9c-a43a-55475bf04165 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:37.477491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2ac41-ed7e-4b9c-a43a-55475bf04165 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:37.505083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:37.513617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:37.514435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.514724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.515006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.515696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:37.517064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:37.529565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:37.532133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:37.533197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 442/905] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Jun 23 10:50:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:37.540271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4c00-9752-421d-8b76-ea8289f469ac None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:37.570620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:37.649742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4c00-9752-421d-8b76-ea8289f469ac None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:37.686011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:37.686679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.686967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.687277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.687996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:37.688307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:37.689688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:37.695015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 464/906] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:37.700974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:37.805833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba19a2f-b24f-45b1-af55-98a9cc429494 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:37.840722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:37.841733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:37.842228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:37.842715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:37.843720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:37.844236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:37.845793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:37.852390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 443/907] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:37.858307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:37.963368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:37.973043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1405ed4-d706-44d9-8f41-f542a373defe None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.008107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5ebd6-7ed7-4f51-b41c-0d1bca133541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.009175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5ebd6-7ed7-4f51-b41c-0d1bca133541 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.009801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5ebd6-7ed7-4f51-b41c-0d1bca133541 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.010088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5ebd6-7ed7-4f51-b41c-0d1bca133541 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.010889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5ebd6-7ed7-4f51-b41c-0d1bca133541 None admin] RBAC: Adding query filter params (interface=public, service_id=63fc348bec96483c802dfa833ef1f124, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:38.011490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5ebd6-7ed7-4f51-b41c-0d1bca133541 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.014391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5ebd6-7ed7-4f51-b41c-0d1bca133541 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.019980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 465/908] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:37 2022] GET /identity/v3/endpoints?service_id=63fc348bec96483c802dfa833ef1f124&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:38.025417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.085418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:38.132248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc141c-1c6f-430d-b133-33a59b27f530 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.167753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881a072-5913-4b16-be45-be92ab51ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.168429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881a072-5913-4b16-be45-be92ab51ac3a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.168708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881a072-5913-4b16-be45-be92ab51ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.168992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881a072-5913-4b16-be45-be92ab51ac3a None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.169680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881a072-5913-4b16-be45-be92ab51ac3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.171048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881a072-5913-4b16-be45-be92ab51ac3a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.177347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 444/909] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:50:38.580388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2730bdf-d4ef-4f7f-b22c-d34998928246 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.580751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2730bdf-d4ef-4f7f-b22c-d34998928246 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.580969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2730bdf-d4ef-4f7f-b22c-d34998928246 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.582591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 466/910] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:38.588885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a12c-b5a4-4f02-92f3-2ac603abc619 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.589178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a12c-b5a4-4f02-92f3-2ac603abc619 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.589467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a12c-b5a4-4f02-92f3-2ac603abc619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.607339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2382a12c-b5a4-4f02-92f3-2ac603abc619 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:38.608544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2382a12c-b5a4-4f02-92f3-2ac603abc619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:38.684137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 445/911] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:38.708079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2730bdf-d4ef-4f7f-b22c-d34998928246 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.819817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2730bdf-d4ef-4f7f-b22c-d34998928246 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:38.856705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:38.857746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:38.858181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:38.858627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:38.859471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:38.861018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:38.873773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:38.875036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 10:50:38.875975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:38.877136 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 467/912] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:38.883616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a12c-b5a4-4f02-92f3-2ac603abc619 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:38.940825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:38.997770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a12c-b5a4-4f02-92f3-2ac603abc619 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.033578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908858c-36d2-45e3-8f2a-9c220c77019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.034990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908858c-36d2-45e3-8f2a-9c220c77019d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.035496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908858c-36d2-45e3-8f2a-9c220c77019d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.035975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908858c-36d2-45e3-8f2a-9c220c77019d None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.036896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908858c-36d2-45e3-8f2a-9c220c77019d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:39.037399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908858c-36d2-45e3-8f2a-9c220c77019d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.039098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908858c-36d2-45e3-8f2a-9c220c77019d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.044947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 446/913] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:39.052232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.076571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.157748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60cf34-d33c-49f3-b743-336c8bc90b58 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.193269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c5547-95d2-46e6-a426-f058672c5fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.194106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c5547-95d2-46e6-a426-f058672c5fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.194392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c5547-95d2-46e6-a426-f058672c5fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.194689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c5547-95d2-46e6-a426-f058672c5fc2 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.195816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c5547-95d2-46e6-a426-f058672c5fc2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:39.196180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c5547-95d2-46e6-a426-f058672c5fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.197038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0a66d-8e8a-41c8-89b9-c29e890afe2a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.197523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0a66d-8e8a-41c8-89b9-c29e890afe2a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.197637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c5547-95d2-46e6-a426-f058672c5fc2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.198183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0a66d-8e8a-41c8-89b9-c29e890afe2a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.200644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 447/914] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:39.205670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 468/915] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:39.206117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbef2a5-7409-433b-8791-c953c6a66dd7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.206500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbef2a5-7409-433b-8791-c953c6a66dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.206785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbef2a5-7409-433b-8791-c953c6a66dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.210648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ddc98-2513-4c5f-9c94-488186a3b001 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.211135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ddc98-2513-4c5f-9c94-488186a3b001 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.211378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ddc98-2513-4c5f-9c94-488186a3b001 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.212667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 469/916] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:39.213355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ddc98-2513-4c5f-9c94-488186a3b001 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.232394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-dbbef2a5-7409-433b-8791-c953c6a66dd7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:39.233908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dbbef2a5-7409-433b-8791-c953c6a66dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:39.302563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.311289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 448/917] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:39.311289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e34e76-452a-4cac-aafd-aea0f946ced9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.311289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e34e76-452a-4cac-aafd-aea0f946ced9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.311289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e34e76-452a-4cac-aafd-aea0f946ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.321999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ddc98-2513-4c5f-9c94-488186a3b001 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.335702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-77e34e76-452a-4cac-aafd-aea0f946ced9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:39.337225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-77e34e76-452a-4cac-aafd-aea0f946ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:39.360677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.361363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.361656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.361939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.362692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.364111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.383742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.410081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 449/918] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:39.410866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e34e76-452a-4cac-aafd-aea0f946ced9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.475229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 470/919] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:39.475229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.533545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e34e76-452a-4cac-aafd-aea0f946ced9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.583928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.585627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.585988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.586361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.602869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:39.606468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cf198-2017-43f4-a416-f3e009a95e19 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:39.613256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:39.614268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:39.615129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:39.616051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 450/920] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:39.621875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.650115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.650906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.651238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.651518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.652248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.653637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.657586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.717358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:39.725486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 471/921] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:39 2022] POST /identity/v3/services => generated 241 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:39.735813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab7b3-0bb1-4713-81da-b24999dd0360 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.790645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.791813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.792326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.792809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.793859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:39.794382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.796114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:39.807504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 451/922] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:39.819018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:39.941538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22b1d6-45bb-4a50-8383-4c7d367d642c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:39.977356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:39.978280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:39.978561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:39.978910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:39.991300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:39.992755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:40.010314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:40.011382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:40.012286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:40.012286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:40.012286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:40.012286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:40.015279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 472/923] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:40.017496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.113961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.127131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf20950-6362-4361-a08f-c6e6c923416b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.164145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4cb22-d503-4686-a594-d25cee895ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.164812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4cb22-d503-4686-a594-d25cee895ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.165096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4cb22-d503-4686-a594-d25cee895ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.165372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4cb22-d503-4686-a594-d25cee895ef8 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.166099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4cb22-d503-4686-a594-d25cee895ef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.166405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4cb22-d503-4686-a594-d25cee895ef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.167801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4cb22-d503-4686-a594-d25cee895ef8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.184722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 452/924] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:40.186057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.239575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.292326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e91f71-acc2-4e27-9fe6-a298105a9bf9 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.300597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cb527-d248-45bb-ba57-92f1ecbc23de None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.300597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cb527-d248-45bb-ba57-92f1ecbc23de None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.300597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cb527-d248-45bb-ba57-92f1ecbc23de None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.306320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 453/925] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:40.308050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea511e-393a-4ca1-b116-08f6110bc102 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.308342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea511e-393a-4ca1-b116-08f6110bc102 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.308625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea511e-393a-4ca1-b116-08f6110bc102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.326037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-71ea511e-393a-4ca1-b116-08f6110bc102 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:40.327426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-71ea511e-393a-4ca1-b116-08f6110bc102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:40.332300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.333400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.333949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.334468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.342695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.344432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:40.360597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:40.361593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:40.362564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:40.365283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 473/926] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:40.371482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.415312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 454/927] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:40.439853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea511e-393a-4ca1-b116-08f6110bc102 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.492282 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963deb2c-4c2e-4468-befb-5fcf0a3b9260 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.504881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.535391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.535585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.536438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.536725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.538497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.539549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.540239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.541954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.559046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea511e-393a-4ca1-b116-08f6110bc102 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.564013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 474/928] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:40.572194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.598477 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.599553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.600026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.600458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.601392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.602934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:40.623435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:40.624434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:40.626839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 455/929] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:40.633336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.685986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc03afa-d92d-4ffa-ad2d-4f5b7566fa66 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.724300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.725366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.725662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.725997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.736468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.738018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.745352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30627ff-9f05-4d5a-8dba-69120bb41f0f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:40.757952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:40.758935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:40.759894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:40.759894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:40.759894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:40.759894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:40.764090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 475/930] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:40.765052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.786514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.787731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.788249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.788724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.789633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.790163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.791762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.795552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.801929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 456/931] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:40.804165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.813990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:40.892764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49766a50-0408-41d6-8a44-7abb66f6236f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.914565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c46a4-8c4a-4e4a-b4da-cff74507bf74 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:40.931517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.932190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.932476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.932752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.933650 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.933966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.935912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.955351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 476/932] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:40.960209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:40.965510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:40.966189 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:40.966475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:40.966760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:40.967519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:40.967828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:40.969200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:40.979896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 457/933] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:40.982676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.063667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:41.064901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:41.084000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe90029-77c1-443d-8499-bfb703c86a15 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.098340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e520a4-9e18-4806-8ee7-89c9846c3ede None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.121939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.123042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.123613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.124115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.131332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:41.133146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:41.134576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.135288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.135577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.135858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.136605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:41.137949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:41.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:41.151147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:41.152277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 477/934] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:41.158276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.240177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 458/935] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:41.269112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad9543-2c05-4950-90ac-af81c10a3589 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.315108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c7c02-114b-4996-8601-0a50d79efc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.315869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c7c02-114b-4996-8601-0a50d79efc19 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.316154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c7c02-114b-4996-8601-0a50d79efc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.316428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c7c02-114b-4996-8601-0a50d79efc19 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.317227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c7c02-114b-4996-8601-0a50d79efc19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:41.317535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c7c02-114b-4996-8601-0a50d79efc19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:41.319325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c7c02-114b-4996-8601-0a50d79efc19 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:41.322544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:41.335077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 478/936] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:41.344353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.461579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf839da-9128-4141-be86-50e82f9dd9b3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:41.489098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:41.499506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd10c50-be30-4a74-b278-b40d14dbd2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:41.500799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd10c50-be30-4a74-b278-b40d14dbd2a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.501255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd10c50-be30-4a74-b278-b40d14dbd2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.501755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd10c50-be30-4a74-b278-b40d14dbd2a5 None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/fef3e61b2a6742ddba146be546996833/roles/3103c74eb4f5404dbb37aaba54d4aac4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.504324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd10c50-be30-4a74-b278-b40d14dbd2a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=fef3e61b2a6742ddba146be546996833, role_id=3103c74eb4f5404dbb37aaba54d4aac4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:41.505970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd10c50-be30-4a74-b278-b40d14dbd2a5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:41.575536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 459/937] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:50:41 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/fef3e61b2a6742ddba146be546996833/roles/3103c74eb4f5404dbb37aaba54d4aac4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:50:41.754065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c618e5-b00d-4b1e-ab06-255801bb02e4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.754607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c618e5-b00d-4b1e-ab06-255801bb02e4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.754607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c618e5-b00d-4b1e-ab06-255801bb02e4 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.756709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 479/938] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:41.763216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2786419-fdca-4a5d-85be-c3252289fefb None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.763216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2786419-fdca-4a5d-85be-c3252289fefb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.763216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2786419-fdca-4a5d-85be-c3252289fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.780744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e2786419-fdca-4a5d-85be-c3252289fefb None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:41.782155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e2786419-fdca-4a5d-85be-c3252289fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:41.852689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 460/939] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:41.875687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c618e5-b00d-4b1e-ab06-255801bb02e4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:41.907599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd8136-959a-4f0b-ac91-74fa5b10c4aa None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.907894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd8136-959a-4f0b-ac91-74fa5b10c4aa None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.908183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd8136-959a-4f0b-ac91-74fa5b10c4aa None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.909823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 461/940] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:41.915549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d9f49-d465-457c-a928-91e775d6cb7e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:41.915844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d9f49-d465-457c-a928-91e775d6cb7e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:41.916133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d9f49-d465-457c-a928-91e775d6cb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:41.934437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-103d9f49-d465-457c-a928-91e775d6cb7e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:41.936129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-103d9f49-d465-457c-a928-91e775d6cb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:41.987652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c618e5-b00d-4b1e-ab06-255801bb02e4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.017770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 462/941] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:42.034603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.035543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.035974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.036413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.037283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.038930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.041085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d9f49-d465-457c-a928-91e775d6cb7e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.043607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING keystone.server.flask.common [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 10:50:42.094227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.103506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.sql.core [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 10:50:42.103506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 10:50:42.103506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [parameters: {'id': '1b151acb17b04004ba347301c31217f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 10:50:42.103506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80343) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b151acb17b04004ba347301c31217f0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b151acb17b04004ba347301c31217f0. Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:42.128232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 10:50:42.129157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application yield resource Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise self.value Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application session.commit() Jun 23 10:50:42.130158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.flush() Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application compat.raise_( Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 10:50:42.131308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 10:50:42.132244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:42.133052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [parameters: {'id': '1b151acb17b04004ba347301c31217f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.136124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:42.138897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b151acb17b04004ba347301c31217f0. Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 480/942] 10.222.0.43 () {60 vars in 1140 bytes} [Thu Jun 23 10:50:41 2022] POST /identity/v3/projects => generated 262 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:42.141192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.169262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d9f49-d465-457c-a928-91e775d6cb7e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.214752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.215255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.215632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.215933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.216469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.217298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.218989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:42.236272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:42.237300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:42.238151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:42.243252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 463/943] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:42.244469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.283567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103990-ac5b-46f8-98a6-31e7d43a5aac None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.332126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.334119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.335315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.335978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.349242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.351680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:42.372832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:42.373903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:42.374830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:42.375296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf8cea-5ba3-45fd-a9b2-4db64088f8c4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.378600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 481/944] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:42.389147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.389313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.414115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.414788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.415202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.415525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.416429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.416758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.418305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.427670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 464/945] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:42.430492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.517527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a4c80-7437-407f-9a2c-eefd270d26cb None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.533593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.557189 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4573d5-16d9-4d70-8f34-e26b1eb07c36 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.557370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.558266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.558838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.559331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.560493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.561007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.562630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.586574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 482/946] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:42.603220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.603220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.603220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.603220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.603673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.603673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.604823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.611406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 465/947] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:42.618162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.740915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe5151-1b26-46a4-8fd2-3ff9f8cf6d89 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.783094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e88827-e0b3-4a06-b285-afde05a164dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.784582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e88827-e0b3-4a06-b285-afde05a164dc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.784874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e88827-e0b3-4a06-b285-afde05a164dc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.785155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e88827-e0b3-4a06-b285-afde05a164dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.785966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e88827-e0b3-4a06-b285-afde05a164dc None admin] RBAC: Adding query filter params (interface=public, service_id=a213caacff5d4d9e87ce608c7578fe59, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:42.786285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e88827-e0b3-4a06-b285-afde05a164dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.787743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e88827-e0b3-4a06-b285-afde05a164dc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.793821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 483/948] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/endpoints?service_id=a213caacff5d4d9e87ce608c7578fe59&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:42.798932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:42.844821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:42.909307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4019c3-0b5e-42db-a845-52d54dd4ab12 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:42.949675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5cb65-1495-406f-bbce-266b9d381656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:42.950350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5cb65-1495-406f-bbce-266b9d381656 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:42.950680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5cb65-1495-406f-bbce-266b9d381656 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:42.950970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5cb65-1495-406f-bbce-266b9d381656 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:42.951678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5cb65-1495-406f-bbce-266b9d381656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:42.953042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5cb65-1495-406f-bbce-266b9d381656 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:42.960140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 466/949] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:42 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 10:50:43.512230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276633b-81c9-46a2-900f-74aa3f46c873 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.512686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276633b-81c9-46a2-900f-74aa3f46c873 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.513067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276633b-81c9-46a2-900f-74aa3f46c873 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.514797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 484/950] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:43.519974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c039e-8840-4bb0-8be3-a32335e459eb None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.520444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c039e-8840-4bb0-8be3-a32335e459eb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.520729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c039e-8840-4bb0-8be3-a32335e459eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.538062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d09c039e-8840-4bb0-8be3-a32335e459eb None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:43.539269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d09c039e-8840-4bb0-8be3-a32335e459eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:43.609325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 467/951] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:43.631945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276633b-81c9-46a2-900f-74aa3f46c873 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.669479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.693962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4eb80-deb8-4ee9-8981-fdf9126b3e34 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.694310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4eb80-deb8-4ee9-8981-fdf9126b3e34 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.694522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4eb80-deb8-4ee9-8981-fdf9126b3e34 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.695991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 468/952] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:43.704857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e66ee-7b73-4cc3-8a72-7a97bd7900a1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.705397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e66ee-7b73-4cc3-8a72-7a97bd7900a1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.705870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e66ee-7b73-4cc3-8a72-7a97bd7900a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.733730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-324e66ee-7b73-4cc3-8a72-7a97bd7900a1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:43.735637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-324e66ee-7b73-4cc3-8a72-7a97bd7900a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:43.754387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276633b-81c9-46a2-900f-74aa3f46c873 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.768474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.792347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:43.793790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:43.794226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:43.794662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:43.795500 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:43.797012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:43.810861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:43.812253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:43.813391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:43.821104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 485/954] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:43.821415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 469/954] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:43.822321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.847333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e66ee-7b73-4cc3-8a72-7a97bd7900a1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:43.950951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:43.955688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e43108-7b77-4013-ab6b-2196a8ca6859 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:43.984038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e66ee-7b73-4cc3-8a72-7a97bd7900a1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.008889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.010175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.010897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.011403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.012760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:44.013399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.015160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.026563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 486/955] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:44.027638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.030803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.032331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.032331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.032533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.044044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:44.052467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:44.053612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:44.054674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:44.061416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 470/956] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:44.062469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.091819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.150800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f595f-10c2-47b4-92a5-5618516e3e42 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.190707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.190707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.190707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.191275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace81a-833a-4f07-91fb-ea55a36077e2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.191361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.191361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:44.191361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.191361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.198026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 487/957] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:44.239475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.240607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.241070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.241452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.242397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:44.242859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.244443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.254034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 471/958] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:44.260491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.294238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.386235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44527b06-a440-4954-8ff2-e4d0b38ac55f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.422611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.423494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.423779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.424103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.434662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.436138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:44.454197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:44.455154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:44.456104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:44.456104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:44.456104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:44.456104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:44.456104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 488/959] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:44.462152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.549889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.566334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6deef91-c0ff-4c23-96f8-7eab396c5aa9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.602268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.602941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.603254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.603538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.604263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:44.604572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.605932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.618517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 472/960] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:44.624846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.718653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:44.738613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d7647-ce39-4059-b8ac-a3df7ff97593 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.777413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1504c-fdb8-46e2-a82f-5a78df633683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.778232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1504c-fdb8-46e2-a82f-5a78df633683 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.778512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1504c-fdb8-46e2-a82f-5a78df633683 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.778784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1504c-fdb8-46e2-a82f-5a78df633683 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.786894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1504c-fdb8-46e2-a82f-5a78df633683 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.788739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1504c-fdb8-46e2-a82f-5a78df633683 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-14f1504c-fdb8-46e2-a82f-5a78df633683 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:44.804493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:44.805904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:44.806884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 489/961] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:44.811599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.812936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fdbd-8d17-43db-8c80-c8fdc98aeb1a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.813234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fdbd-8d17-43db-8c80-c8fdc98aeb1a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.813525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fdbd-8d17-43db-8c80-c8fdc98aeb1a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.815149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 490/962] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:44.820805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7a9b-812a-400d-8d3e-2fd75458a49e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.821093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7a9b-812a-400d-8d3e-2fd75458a49e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.821374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7a9b-812a-400d-8d3e-2fd75458a49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.841682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ff2d7a9b-812a-400d-8d3e-2fd75458a49e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:44.843817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ff2d7a9b-812a-400d-8d3e-2fd75458a49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:44.931615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4888f4-67fb-4f13-ab02-c1085d505433 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:44.947498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 491/963] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:44.973287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d7a9b-812a-400d-8d3e-2fd75458a49e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:44.985964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:44.987385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:44.987888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:44.988690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:44.990099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:44.990843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:44.993312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.022530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 473/964] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 10:50:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:45.027948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.031547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.075039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.107847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d7a9b-812a-400d-8d3e-2fd75458a49e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.147088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.147960 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.148322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.148578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.149220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.150661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.153539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d67224-5e1c-426a-8bcc-14a4d7226658 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.188008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edead5b6-0c35-4f94-8eec-3872d6ecbd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.188658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edead5b6-0c35-4f94-8eec-3872d6ecbd7e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.188948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edead5b6-0c35-4f94-8eec-3872d6ecbd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.189231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edead5b6-0c35-4f94-8eec-3872d6ecbd7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.194075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.sql.core [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 10:50:45.194075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 10:50:45.194075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [parameters: {'user_id': 'a3ef8a2a1f6c4e84b41521cda9857ee5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 10:50:45.194075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80343) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-d04f0062-e416-4e97-88c2-748cd00e414b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:45.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:45.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application yield resource Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application next(self.gen) Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise self.value Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application session.commit() Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 10:50:45.212172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.session.flush() Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application compat.raise_( Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 10:50:45.213275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 10:50:45.214227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application conn.query(q) Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application result.read() Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 10:50:45.215086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a3ef8a2a1f6c4e84b41521cda9857ee5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:45.215997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 10:50:45.218247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 492/965] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 10:50:44 2022] POST /identity/v3/users => generated 160 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d0d72-8d1d-4ef6-a874-f0dbd8b8dfc5 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d0d72-8d1d-4ef6-a874-f0dbd8b8dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.221538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d0d72-8d1d-4ef6-a874-f0dbd8b8dfc5 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.228034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 493/966] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:45.228608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d0d72-8d1d-4ef6-a874-f0dbd8b8dfc5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.267233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edead5b6-0c35-4f94-8eec-3872d6ecbd7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.268688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edead5b6-0c35-4f94-8eec-3872d6ecbd7e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.270576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.338971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d0d72-8d1d-4ef6-a874-f0dbd8b8dfc5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.371195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.376795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.377600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.377889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.378339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.387264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 474/967] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:50:45.387264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7709c-ecb6-4c0f-9037-e290c6cd524e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.387264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7709c-ecb6-4c0f-9037-e290c6cd524e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.387264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7709c-ecb6-4c0f-9037-e290c6cd524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.397301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.399063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.401625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fbf7709c-ecb6-4c0f-9037-e290c6cd524e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:45.402827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fbf7709c-ecb6-4c0f-9037-e290c6cd524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:45.415893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:45.417139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 10:50:45.418156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.426714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 494/968] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:45.428453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.480505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 475/969] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:45.481715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7709c-ecb6-4c0f-9037-e290c6cd524e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.522960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.549911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a82ba-7bdf-41f0-8a44-11630e544d6c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.593505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.594221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.594515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.594801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.597167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7709c-ecb6-4c0f-9037-e290c6cd524e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.606114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.607591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:45.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:45.625531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:45.626478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:45.626478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:45.626478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:45.626478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.630609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 495/970] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:45.632214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.636060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.636739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.637028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.637306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.638109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:45.638422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.639884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.675142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 476/971] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:45.676019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.685015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.767430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9253f-f7cf-4155-8e7b-8e2cd565fc4b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.790946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.801791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e774266-af9a-465e-ac20-f1592569fbfa None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.811596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.813127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.813617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.814111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.814984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.816642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:45.832328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:45.833417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:45.834362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:45.837161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 496/972] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:45.844424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.846535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:45.847302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:45.847596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:45.848036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:45.848878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:45.849197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:45.850647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:45.868889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 477/973] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:45.871208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:45.972630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b09b-2979-4173-886e-50af12c076fc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:45.984187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:45.989469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dcfda-5adf-4f47-90a6-4a7d326b9399 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.010953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.011696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.011987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.012268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.012984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:46.013293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.014662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.019817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 497/974] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:46.030097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.030698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.031465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.031762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.032046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.040431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.041916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:46.061270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:46.062222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:46.063146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 478/975] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:46.069521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.082605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.141369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf14f7a-5a49-44cd-a8f7-cc77c52ad208 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.172841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572fa42-4702-499e-b428-523b74f24a5a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.178341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.179099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.179475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.179826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.180603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:46.181398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.183232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.193996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 498/976] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:46.194900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.209745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.210535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.210928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.211241 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.212050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:46.212368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.213807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.231989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 479/977] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:46.233982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.238031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.308297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.311265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6a052-d9cb-46c1-9d36-bfbcb99714d8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.340544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7466504-6068-415a-a2c8-48eb4902544e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.351735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.352444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.352734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.353013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.353760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.355209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.376203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6d13f-e4e9-4a05-8ffe-4395e1485938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.376908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6d13f-e4e9-4a05-8ffe-4395e1485938 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.377197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6d13f-e4e9-4a05-8ffe-4395e1485938 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.377476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6d13f-e4e9-4a05-8ffe-4395e1485938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.378339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6d13f-e4e9-4a05-8ffe-4395e1485938 None admin] RBAC: Adding query filter params (role.id=3103c74eb4f5404dbb37aaba54d4aac4, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=fef3e61b2a6742ddba146be546996833) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:46.378654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6d13f-e4e9-4a05-8ffe-4395e1485938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.380124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6d13f-e4e9-4a05-8ffe-4395e1485938 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.386575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 480/978] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:46 2022] GET /identity/v3/role_assignments?user.id=fef3e61b2a6742ddba146be546996833&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=3103c74eb4f5404dbb37aaba54d4aac4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:46.387786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162cb6d-96a1-4277-8d56-cd61ef9a7819 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.388075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162cb6d-96a1-4277-8d56-cd61ef9a7819 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.388420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162cb6d-96a1-4277-8d56-cd61ef9a7819 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.389626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 481/979] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:46.398767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fdd42-e675-4eac-8079-2e363998588f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.398767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fdd42-e675-4eac-8079-2e363998588f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.398767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fdd42-e675-4eac-8079-2e363998588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.414539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b51fdd42-e675-4eac-8079-2e363998588f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:46.415819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b51fdd42-e675-4eac-8079-2e363998588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:46.425915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 499/980] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:46.514768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 482/981] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:46.538929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:46.641384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:46.651737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecee48-2075-4414-babf-0d1ad72a0031 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:46.688185 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9ada2-1481-46c6-97d0-9ef49f5e2d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:46.688863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ada2-1481-46c6-97d0-9ef49f5e2d03 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:46.689153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ada2-1481-46c6-97d0-9ef49f5e2d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:46.689436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9ada2-1481-46c6-97d0-9ef49f5e2d03 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:46.690179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9ada2-1481-46c6-97d0-9ef49f5e2d03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:46.691596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9ada2-1481-46c6-97d0-9ef49f5e2d03 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:46.819536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 500/982] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:46 2022] POST /identity/v3/services => generated 242 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:48.489901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93788dfc-6ffb-4cae-a6b5-9dd0a40c51fc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.490432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93788dfc-6ffb-4cae-a6b5-9dd0a40c51fc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.490802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93788dfc-6ffb-4cae-a6b5-9dd0a40c51fc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.492247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 483/983] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:48.497367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915af2c-0936-4d02-b77c-8d4fe66d0cb1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.497778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915af2c-0936-4d02-b77c-8d4fe66d0cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.497976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915af2c-0936-4d02-b77c-8d4fe66d0cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.500834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ef998-86e5-414a-a876-402cf8beca1c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.501133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ef998-86e5-414a-a876-402cf8beca1c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.501420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ef998-86e5-414a-a876-402cf8beca1c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.502973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 484/984] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:48.508612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2c91-1307-458e-be15-43760b20299d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.508905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2c91-1307-458e-be15-43760b20299d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.509186 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2c91-1307-458e-be15-43760b20299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.516736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f915af2c-0936-4d02-b77c-8d4fe66d0cb1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:48.518259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f915af2c-0936-4d02-b77c-8d4fe66d0cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:48.525565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0bae2c91-1307-458e-be15-43760b20299d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:48.526896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0bae2c91-1307-458e-be15-43760b20299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:48.588940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.597935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 501/985] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:48.607223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 485/986] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:48.620739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915af2c-0936-4d02-b77c-8d4fe66d0cb1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.629074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae2c91-1307-458e-be15-43760b20299d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.702305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.726204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915af2c-0936-4d02-b77c-8d4fe66d0cb1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.733978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae2c91-1307-458e-be15-43760b20299d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.763723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.764921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.765357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.765782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] PATH_INFO: `/v3/services/image` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.766588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.768330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.772748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.773550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.791709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.791709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.791709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.791709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:48.792061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:48.793043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:48.793886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 502/987] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:48.794691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:48.795647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:48.796603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:48.797509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.797594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 486/988] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:48.798557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.859023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.899201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614a533-5e07-4886-a21c-2c68aeaf2199 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.913381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944cf25-6ebc-44c8-b1c5-3b85d8603582 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:48.939556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e5ebe-95a0-4bd1-8c6b-c260a96db78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.940222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5ebe-95a0-4bd1-8c6b-c260a96db78c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.940511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5ebe-95a0-4bd1-8c6b-c260a96db78c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.940807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5ebe-95a0-4bd1-8c6b-c260a96db78c None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.941510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e5ebe-95a0-4bd1-8c6b-c260a96db78c None admin] RBAC: Adding query filter params (name=image) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:48.941818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e5ebe-95a0-4bd1-8c6b-c260a96db78c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.943221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e5ebe-95a0-4bd1-8c6b-c260a96db78c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.948784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 503/989] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:48.949963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7973213-5af4-4117-86f9-2fbcd929c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:48.950172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdddd00-91d0-4d4f-a91c-168cc36053c9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.950434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdddd00-91d0-4d4f-a91c-168cc36053c9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.950633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7973213-5af4-4117-86f9-2fbcd929c2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.950728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdddd00-91d0-4d4f-a91c-168cc36053c9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.950923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7973213-5af4-4117-86f9-2fbcd929c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.951244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7973213-5af4-4117-86f9-2fbcd929c2f2 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.952127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7973213-5af4-4117-86f9-2fbcd929c2f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:48.952271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 504/990] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:48.952476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7973213-5af4-4117-86f9-2fbcd929c2f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:48.953613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7973213-5af4-4117-86f9-2fbcd929c2f2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:48.956345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdddd00-91d0-4d4f-a91c-168cc36053c9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:48.958573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 487/991] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:48.959839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e14ead-9e84-4799-b2f3-f976d6acc2c5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:48.960134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e14ead-9e84-4799-b2f3-f976d6acc2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:48.960420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e14ead-9e84-4799-b2f3-f976d6acc2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:48.965597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:48.977235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-78e14ead-9e84-4799-b2f3-f976d6acc2c5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:48.978465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-78e14ead-9e84-4799-b2f3-f976d6acc2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:49.045945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 488/992] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:49.046712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e14ead-9e84-4799-b2f3-f976d6acc2c5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.061106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdddd00-91d0-4d4f-a91c-168cc36053c9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.096528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.097263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.097554 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.097839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.098552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] RBAC: Adding query filter params (type=image) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:49.098868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.099602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:49.100779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.144954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 505/993] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:49.145887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.150889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e14ead-9e84-4799-b2f3-f976d6acc2c5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.190983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.191715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.192009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.192295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.193035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:49.193967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.195633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.201662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 489/994] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:49.207895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.230823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:49.267524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf450c1-eb4d-4014-9c6e-f765edea1f87 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.309164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.309905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.310191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.310472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.311132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.313020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad84083-b1c6-451b-ad03-124b855ab221 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:49.328075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:49.329106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:49.329974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 506/995] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:49.336005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.341736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:49.364579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.366338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.366338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.366742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.367466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] RBAC: Adding query filter params (interface=admin, service_id=a213caacff5d4d9e87ce608c7578fe59, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:49.367787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.369163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.374333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 490/996] 10.222.0.43 () {56 vars in 1249 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/endpoints?service_id=a213caacff5d4d9e87ce608c7578fe59&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:49.380167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.454065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe20d21-218a-42ad-a524-75de4a695053 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.491700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.492691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.492981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.493262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.493982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:49.494292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.495698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.496141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4368bb0-7a64-4bcc-85b4-20844d069979 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.496981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:49.506249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 507/997] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:49.513321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.538966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.539673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.540315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.540599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.541542 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.543104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.551281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 491/998] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 10:50:49.619517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94889df3-c7c3-4454-b166-c03458c0f83e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.656465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.657147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.657432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.657712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.658442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:49.658760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.660196 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.671331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 508/999] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:49.672454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.717901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:49.779444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c998139-1700-438d-a555-1d54b5984f66 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.813901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69543a-c91d-44db-92db-df94bbed3c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.814567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69543a-c91d-44db-92db-df94bbed3c47 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.814854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69543a-c91d-44db-92db-df94bbed3c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.815134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69543a-c91d-44db-92db-df94bbed3c47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.815957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69543a-c91d-44db-92db-df94bbed3c47 None admin] RBAC: Adding query filter params (interface=public, service_id=5cc00b61d0c74d3280b2771a2cdbed6e, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:49.816278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69543a-c91d-44db-92db-df94bbed3c47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.817645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69543a-c91d-44db-92db-df94bbed3c47 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.827695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 492/1000] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/endpoints?service_id=5cc00b61d0c74d3280b2771a2cdbed6e&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:49.828982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:49.893154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:49.936728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52de6c-3901-4436-83eb-378db29cb166 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:49.972786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8588d-5103-4367-90da-be5ec808c027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:49.973453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8588d-5103-4367-90da-be5ec808c027 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:49.973740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8588d-5103-4367-90da-be5ec808c027 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:49.974016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8588d-5103-4367-90da-be5ec808c027 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:49.975057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8588d-5103-4367-90da-be5ec808c027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:49.976246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8588d-5103-4367-90da-be5ec808c027 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:49.991213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 509/1001] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:49 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:50:51.189321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04049039-a972-48f5-984c-32bdf3e4bdb1 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:51.189843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04049039-a972-48f5-984c-32bdf3e4bdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:51.189843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04049039-a972-48f5-984c-32bdf3e4bdb1 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:51.191315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 493/1002] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:51.196287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dfa4e-17f9-485a-8f7e-c70a11c47644 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:51.196705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dfa4e-17f9-485a-8f7e-c70a11c47644 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:51.196887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dfa4e-17f9-485a-8f7e-c70a11c47644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:51.214268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-546dfa4e-17f9-485a-8f7e-c70a11c47644 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:51.215458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-546dfa4e-17f9-485a-8f7e-c70a11c47644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:51.281912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 510/1003] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:51.304518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04049039-a972-48f5-984c-32bdf3e4bdb1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:51.406418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04049039-a972-48f5-984c-32bdf3e4bdb1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:51.408778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:51.440465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef3d71-3e42-4522-93f7-1f4228999319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:51.441233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef3d71-3e42-4522-93f7-1f4228999319 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:51.441807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef3d71-3e42-4522-93f7-1f4228999319 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:51.442100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef3d71-3e42-4522-93f7-1f4228999319 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:51.442837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef3d71-3e42-4522-93f7-1f4228999319 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:51.444827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef3d71-3e42-4522-93f7-1f4228999319 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:51.563202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 494/1004] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:51 2022] POST /identity/v3/services => generated 242 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:51.601474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfe7c-a98d-4305-9d11-3ec19e06b908 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:51.601799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfe7c-a98d-4305-9d11-3ec19e06b908 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:51.602048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfe7c-a98d-4305-9d11-3ec19e06b908 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:51.603621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 511/1005] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:51.609065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad896f8-650f-4fe4-946e-7df19fa181cf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:51.609355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad896f8-650f-4fe4-946e-7df19fa181cf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:51.609639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad896f8-650f-4fe4-946e-7df19fa181cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:51.627435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3ad896f8-650f-4fe4-946e-7df19fa181cf None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:51.628659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3ad896f8-650f-4fe4-946e-7df19fa181cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:51.710046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 495/1006] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:51 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:51.733007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dfe7c-a98d-4305-9d11-3ec19e06b908 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:51.792812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:51.838337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dfe7c-a98d-4305-9d11-3ec19e06b908 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:51.877965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae788a-ac36-4d6c-bcb4-1c9dc45fd158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:51.878779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae788a-ac36-4d6c-bcb4-1c9dc45fd158 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:51.879065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae788a-ac36-4d6c-bcb4-1c9dc45fd158 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:51.879374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae788a-ac36-4d6c-bcb4-1c9dc45fd158 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:51.880033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae788a-ac36-4d6c-bcb4-1c9dc45fd158 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:51.882064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae788a-ac36-4d6c-bcb4-1c9dc45fd158 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2dae788a-ac36-4d6c-bcb4-1c9dc45fd158 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:51.894362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:51.895598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:51.896492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 512/1007] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 10:50:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:51.902644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad896f8-650f-4fe4-946e-7df19fa181cf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:51.998496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:52.008684 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad896f8-650f-4fe4-946e-7df19fa181cf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:52.042767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:52.043463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.043750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.044028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.044738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:52.045045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:52.046444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:52.049574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef8f0f-f704-4a20-9023-c58180973763 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.049854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef8f0f-f704-4a20-9023-c58180973763 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.050113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef8f0f-f704-4a20-9023-c58180973763 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.051662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 513/1008] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:52.057190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 496/1009] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:52.057324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b935a-1eb9-4340-b0ae-ae12e0602a2e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.057614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b935a-1eb9-4340-b0ae-ae12e0602a2e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.057902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b935a-1eb9-4340-b0ae-ae12e0602a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.058013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:52.076027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1d9b935a-1eb9-4340-b0ae-ae12e0602a2e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:52.077222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1d9b935a-1eb9-4340-b0ae-ae12e0602a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:52.148634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 514/1010] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:52 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:52.167622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7fee2-9f44-41ea-8077-1c3b454ac688 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:52.172213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b935a-1eb9-4340-b0ae-ae12e0602a2e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:52.204589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:52.205529 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.206630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.206919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.207669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:52.207987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:52.209470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:52.219241 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 497/1011] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 10:50:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:52.220479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:52.226370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:52.241104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:52.277093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b935a-1eb9-4340-b0ae-ae12e0602a2e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:52.313368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:52.314581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.315080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.315618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.316466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:52.318017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:52.324992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb4e88-667c-456e-92fa-f83949c6a6d1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:52.332260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:52.333408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 10:50:52.334737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:52.337303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 515/1012] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:52.342802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:52.361225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:52.361893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.362180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.362461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.363226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:52.364602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:52.450449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f2ccf-f8c8-4e97-9bab-189b81421c5a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:52.463972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:52.482027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 498/1013] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:52.493836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:52.494530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.494815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.495094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.496056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:52.496372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:52.497804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:52.510564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 516/1014] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:52.512396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:52.601453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:52.624471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b51ec6-7363-46e4-b5b4-f03b6a25c7f2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:52.662510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f28a5c-3de0-42f5-a987-5759aea4fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:52.663260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f28a5c-3de0-42f5-a987-5759aea4fd31 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.663492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f28a5c-3de0-42f5-a987-5759aea4fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.663771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f28a5c-3de0-42f5-a987-5759aea4fd31 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.664482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f28a5c-3de0-42f5-a987-5759aea4fd31 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:52.664789 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f28a5c-3de0-42f5-a987-5759aea4fd31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:52.666158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f28a5c-3de0-42f5-a987-5759aea4fd31 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:52.671209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 499/1015] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 10:50:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:52.677003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:52.787996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105da1d-d6cc-4947-96d5-0b8adbb7dcb6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:52.823041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6842d6-9ca4-40b7-a5e7-00a39a3f574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:52.823735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6842d6-9ca4-40b7-a5e7-00a39a3f574b None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:52.824019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6842d6-9ca4-40b7-a5e7-00a39a3f574b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:52.824296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6842d6-9ca4-40b7-a5e7-00a39a3f574b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:52.825057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6842d6-9ca4-40b7-a5e7-00a39a3f574b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:52.826415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6842d6-9ca4-40b7-a5e7-00a39a3f574b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:52.904143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 517/1016] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:50:53.741829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38073604-03e2-4cad-8566-32db10613bc9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.742361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38073604-03e2-4cad-8566-32db10613bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.742444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38073604-03e2-4cad-8566-32db10613bc9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.743858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 500/1017] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:53.749042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a2ace-8a48-4fd2-ad2c-d564e7035b61 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:53.749441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a2ace-8a48-4fd2-ad2c-d564e7035b61 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:53.749624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a2ace-8a48-4fd2-ad2c-d564e7035b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:53.766647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bd2a2ace-8a48-4fd2-ad2c-d564e7035b61 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:53.767866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bd2a2ace-8a48-4fd2-ad2c-d564e7035b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:53.793065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:53.844395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 518/1018] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:53.868534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38073604-03e2-4cad-8566-32db10613bc9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:53.980680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38073604-03e2-4cad-8566-32db10613bc9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:54.015305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:54.016023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.016307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.016584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] PATH_INFO: `/v3/services/image` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.017231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:54.018575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:54.030908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:54.031969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:54.032825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:54.035250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 501/1019] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Jun 23 10:50:53 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:54.038326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a2ace-8a48-4fd2-ad2c-d564e7035b61 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:54.143172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a2ace-8a48-4fd2-ad2c-d564e7035b61 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:54.176775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:54.177993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.178275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.178550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.179288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] RBAC: Adding query filter params (name=image) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:54.179599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:54.180962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:54.190416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 519/1020] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:54.191638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:54.201666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:54.293476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58128b-bd58-4865-81c6-02d9e52fb782 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:54.327962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:54.328621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.328908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.329187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.329889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] RBAC: Adding query filter params (type=image) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:54.330197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:54.331581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:54.341407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 502/1021] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:54.342561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:54.364580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:54.444717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644667c-4202-48d3-9434-068136f32f65 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:54.478768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180f480-6436-4f90-ae76-78f0f72d5784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:54.479449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180f480-6436-4f90-ae76-78f0f72d5784 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.479734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180f480-6436-4f90-ae76-78f0f72d5784 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.480006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180f480-6436-4f90-ae76-78f0f72d5784 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.480794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180f480-6436-4f90-ae76-78f0f72d5784 None admin] RBAC: Adding query filter params (interface=public, service_id=42647a069c904a038c8c705a7acfb8bb, region_id=RegionOne) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:54.481104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180f480-6436-4f90-ae76-78f0f72d5784 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:54.482463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180f480-6436-4f90-ae76-78f0f72d5784 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:54.487875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 520/1022] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:50:54 2022] GET /identity/v3/endpoints?service_id=42647a069c904a038c8c705a7acfb8bb&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:54.493801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:54.558785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6b08b-115d-4e29-acfe-8702fa0195bc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.558983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6b08b-115d-4e29-acfe-8702fa0195bc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.559283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6b08b-115d-4e29-acfe-8702fa0195bc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.560775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 521/1023] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:54.567232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55bc58-0def-48ba-a63b-d1b74c6dca01 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.568768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55bc58-0def-48ba-a63b-d1b74c6dca01 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.569486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55bc58-0def-48ba-a63b-d1b74c6dca01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.570021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:54.589884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cb55bc58-0def-48ba-a63b-d1b74c6dca01 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:54.592905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cb55bc58-0def-48ba-a63b-d1b74c6dca01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:54.603299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e794ca-0dc4-4ae8-a013-e6735cdfc642 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:54.641629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:54.642305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.642600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.642884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.643607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:54.645059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:54.652502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 503/1024] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:50:54 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 10:50:54.673729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 522/1025] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:54.698190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:54.810279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c784ad-270e-451f-af74-bb350955d15f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:54.846387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734245e-536a-4121-899c-7aa1987838ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:54.847210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734245e-536a-4121-899c-7aa1987838ff None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:54.847498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734245e-536a-4121-899c-7aa1987838ff None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:54.847776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734245e-536a-4121-899c-7aa1987838ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:54.848942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734245e-536a-4121-899c-7aa1987838ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:54.850411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734245e-536a-4121-899c-7aa1987838ff None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:54.852307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 504/1026] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:54.857836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55bc58-0def-48ba-a63b-d1b74c6dca01 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:54.874555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:54.968326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55bc58-0def-48ba-a63b-d1b74c6dca01 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:55.003887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:55.004553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.004836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.005111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.006734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:55.008205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:55.032191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada2166-e0fa-4174-bfd8-75a9e8066261 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.033153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada2166-e0fa-4174-bfd8-75a9e8066261 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.033153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada2166-e0fa-4174-bfd8-75a9e8066261 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.034307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 505/1027] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:55.039725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a570a-fcb1-44b6-a70e-434424fc80ac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.040013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a570a-fcb1-44b6-a70e-434424fc80ac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.040293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a570a-fcb1-44b6-a70e-434424fc80ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.045281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:55.057748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-be3a570a-fcb1-44b6-a70e-434424fc80ac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:55.058922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-be3a570a-fcb1-44b6-a70e-434424fc80ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:55.119209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 523/1028] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:50:54 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:55.136938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 506/1029] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:55.160879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:55.231442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:55.269080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ce7cc-6dbf-4ec2-b32a-027ac1a33825 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:55.308332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:55.309189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.309441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.309721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.320644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:55.328885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:55.329825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:55.330685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 524/1030] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:55.336293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a570a-fcb1-44b6-a70e-434424fc80ac None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:55.443160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a570a-fcb1-44b6-a70e-434424fc80ac None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:55.472510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:55.477761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:55.478440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.478731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.479010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.479802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:55.480176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:55.481457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:55.494879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 507/1031] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:55.495844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:55.597610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc2e0d-fc7f-4aea-81e6-1c5e613f538f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:55.626364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:55.637223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:55.638028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.638313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.638591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.650044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:55.651698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:55.669177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:55.670126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:55.671063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:55.671063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:55.671063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:55.671063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:55.678156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 525/1032] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:55.679209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:55.783666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b58e58-9de6-438b-abdc-05acabc90373 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:55.817886 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:55.819040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.819343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.819625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.820351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:55.820661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:55.822016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:55.834253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 508/1033] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:55.839641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:55.951795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220c46-d5fc-45fc-8fd1-decad7be50d2 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:55.985330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:55.986118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:55.986437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:55.986716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:55.994549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:55.996007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:50:56.009803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:56.010769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:50:56.012334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:56.015593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 526/1034] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:50:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:56.016789 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.072602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.118177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cc4ba-8ab4-40ff-8f16-8d44c07d75f6 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.151577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.152248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.152536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.152815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.153651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:56.153966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.155411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.174716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 509/1035] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:56.180416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.221476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.285980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a0728-31ba-4294-a8f4-78df58aa328d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.320861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.321514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.321801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.322079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.390594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.391982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.430857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.491210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 527/1036] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:50:56.496641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.594800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76e798-30d7-4a9e-90fd-a9b77ea9bbb4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.608434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.630098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c548f7-6c09-46ae-9739-21ce99a71854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.646046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c548f7-6c09-46ae-9739-21ce99a71854 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.646046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c548f7-6c09-46ae-9739-21ce99a71854 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.646046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c548f7-6c09-46ae-9739-21ce99a71854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.646046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c548f7-6c09-46ae-9739-21ce99a71854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.646046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c548f7-6c09-46ae-9739-21ce99a71854 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-26c548f7-6c09-46ae-9739-21ce99a71854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:50:56.663136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:56.664112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:56.665014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:50:56.665014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:50:56.665014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:50:56.665014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:56.665014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 510/1037] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:56.669929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.704255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4d237-7422-432e-aaf6-1d5a18171f32 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.704549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4d237-7422-432e-aaf6-1d5a18171f32 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.704828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4d237-7422-432e-aaf6-1d5a18171f32 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.706345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 511/1038] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:56.711221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d85d61-3769-4307-baa8-c49b3f8bcae3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.711526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d85d61-3769-4307-baa8-c49b3f8bcae3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.711813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d85d61-3769-4307-baa8-c49b3f8bcae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.730122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e8d85d61-3769-4307-baa8-c49b3f8bcae3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:56.731336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e8d85d61-3769-4307-baa8-c49b3f8bcae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:56.778892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b954be-9f9f-4322-9381-6aa888fbe572 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.801273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 512/1039] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:56.815104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.815802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.816091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.816372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.817100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:56.817413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.819005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.824272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d85d61-3769-4307-baa8-c49b3f8bcae3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.826586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.832744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 528/1040] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:56.838802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:56.935584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:56.942468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d85d61-3769-4307-baa8-c49b3f8bcae3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.957759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc774d7-0c6e-4c64-99a3-6941af8292c1 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:56.978481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:56.979313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:56.979602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:56.979883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] PATH_INFO: `/v3/services/image` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:56.980535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:56.981904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:56.994389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:56.995590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:50:56.996879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 513/1041] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:57.001302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:57.003952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.005835 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.006354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.006804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.015475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.017601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:50:57.036110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:57.037795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:50:57.038987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:57.040694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 529/1042] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:50:57.046079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:57.107682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:57.110409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905820d-af56-4f71-b2c3-a3aa4bda66ca None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:57.147017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.147771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.148056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.148341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.149067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] RBAC: Adding query filter params (name=image) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:57.149369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.150741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.155574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a032a0b-074d-4b81-826c-9cc3ecab8add None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:57.160538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 514/1043] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:57.161501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:57.187703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:57.197395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccf98-46fe-4d5e-a8e6-e045f7c9f95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.198067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccf98-46fe-4d5e-a8e6-e045f7c9f95b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.198358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccf98-46fe-4d5e-a8e6-e045f7c9f95b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.200800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccf98-46fe-4d5e-a8e6-e045f7c9f95b None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.200800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccf98-46fe-4d5e-a8e6-e045f7c9f95b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:57.200800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccf98-46fe-4d5e-a8e6-e045f7c9f95b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.201218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccf98-46fe-4d5e-a8e6-e045f7c9f95b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.215105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 530/1044] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:50:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:57.216446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f1c3-81ce-47c9-95fe-f18abd8ea2b8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.216735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f1c3-81ce-47c9-95fe-f18abd8ea2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.217014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7f1c3-81ce-47c9-95fe-f18abd8ea2b8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.218357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 531/1045] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:57.220953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f1c3-81ce-47c9-95fe-f18abd8ea2b8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:57.267860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2bc35-6587-4f49-96c9-f2d07e733c7f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:57.303312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807eeb3-bb1d-40e2-a2e3-d08598825fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.303982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807eeb3-bb1d-40e2-a2e3-d08598825fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.304257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807eeb3-bb1d-40e2-a2e3-d08598825fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.304534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807eeb3-bb1d-40e2-a2e3-d08598825fa5 None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.305238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807eeb3-bb1d-40e2-a2e3-d08598825fa5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:57.305547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807eeb3-bb1d-40e2-a2e3-d08598825fa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.306904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807eeb3-bb1d-40e2-a2e3-d08598825fa5 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.311702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 515/1046] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:57.311702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cc625-7d26-481a-afe6-d1f2cd6fa187 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.311702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cc625-7d26-481a-afe6-d1f2cd6fa187 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.311702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cc625-7d26-481a-afe6-d1f2cd6fa187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.321276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7f1c3-81ce-47c9-95fe-f18abd8ea2b8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:57.331152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ca8cc625-7d26-481a-afe6-d1f2cd6fa187 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:57.332428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ca8cc625-7d26-481a-afe6-d1f2cd6fa187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:57.338944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:57.355921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.356569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.356855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.357133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.357954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] RBAC: Adding query filter params (role.id=3103c74eb4f5404dbb37aaba54d4aac4, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=54bb42878fff477099bbf56a46e9eec0) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:57.358264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.359712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.365662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 532/1047] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:50:57 2022] GET /identity/v3/role_assignments?user.id=54bb42878fff477099bbf56a46e9eec0&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=3103c74eb4f5404dbb37aaba54d4aac4 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:57.366417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:57.414298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 516/1048] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:50:57.439165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cc625-7d26-481a-afe6-d1f2cd6fa187 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:57.458291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:57.497916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb31476-d29b-465d-819e-dd551b50b6e5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:57.537022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b330e4-1555-454b-a3fd-745a418a82a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.537689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b330e4-1555-454b-a3fd-745a418a82a0 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.537974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b330e4-1555-454b-a3fd-745a418a82a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.538245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b330e4-1555-454b-a3fd-745a418a82a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.539330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b330e4-1555-454b-a3fd-745a418a82a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.540386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b330e4-1555-454b-a3fd-745a418a82a0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.559151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cc625-7d26-481a-afe6-d1f2cd6fa187 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:57.594439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.595522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.595805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.596078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.597286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.598707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.600616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 517/1049] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:50:57.605992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:57.619518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 533/1050] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:50:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:50:57.706891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:57.725837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc776a-3530-4eb1-ad42-8e288254c357 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:57.769441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff49f3-3ee4-4d6b-bea4-d57a3ec2d900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:57.770123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff49f3-3ee4-4d6b-bea4-d57a3ec2d900 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:57.770411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff49f3-3ee4-4d6b-bea4-d57a3ec2d900 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:57.770692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff49f3-3ee4-4d6b-bea4-d57a3ec2d900 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:57.772012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff49f3-3ee4-4d6b-bea4-d57a3ec2d900 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:57.773464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff49f3-3ee4-4d6b-bea4-d57a3ec2d900 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:57.870025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 518/1051] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:50:57 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:50:59.405863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06971dfb-c624-4449-8c0f-9bc826adb0c4 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.406420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06971dfb-c624-4449-8c0f-9bc826adb0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.406420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06971dfb-c624-4449-8c0f-9bc826adb0c4 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.407996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 534/1052] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:50:59.413337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ede-b00b-4585-a64b-ffb1134f0561 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.413758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ede-b00b-4585-a64b-ffb1134f0561 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.413961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ede-b00b-4585-a64b-ffb1134f0561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.431424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c3348ede-b00b-4585-a64b-ffb1134f0561 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:59.432612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c3348ede-b00b-4585-a64b-ffb1134f0561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:59.507999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 519/1053] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:59.531567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06971dfb-c624-4449-8c0f-9bc826adb0c4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:59.573560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:59.635716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06971dfb-c624-4449-8c0f-9bc826adb0c4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:59.671086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c32ea-2b0a-4aa8-a606-4a2833e3a88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:59.671986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c32ea-2b0a-4aa8-a606-4a2833e3a88a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.672277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c32ea-2b0a-4aa8-a606-4a2833e3a88a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.672558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c32ea-2b0a-4aa8-a606-4a2833e3a88a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.682859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c32ea-2b0a-4aa8-a606-4a2833e3a88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b97c32ea-2b0a-4aa8-a606-4a2833e3a88a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:50:59.690953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:50:59.692027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:50:59.693434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:50:59.695509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 535/1054] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 10:50:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:50:59.698834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ede-b00b-4585-a64b-ffb1134f0561 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:59.700121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ef483-c50d-41f4-81c6-1f4b2ec65c1e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.700418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ef483-c50d-41f4-81c6-1f4b2ec65c1e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.700703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ef483-c50d-41f4-81c6-1f4b2ec65c1e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.702279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 536/1055] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:50:59.707049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23cb2d-9f1f-453d-b245-39b4c4052cd9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.707354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23cb2d-9f1f-453d-b245-39b4c4052cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.707649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23cb2d-9f1f-453d-b245-39b4c4052cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.725161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5d23cb2d-9f1f-453d-b245-39b4c4052cd9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:50:59.726371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5d23cb2d-9f1f-453d-b245-39b4c4052cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:50:59.775873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:59.799271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 537/1056] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:50:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:50:59.816845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ede-b00b-4585-a64b-ffb1134f0561 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:50:59.822587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23cb2d-9f1f-453d-b245-39b4c4052cd9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:59.850857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:50:59.856402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:50:59.857143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:50:59.861022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:50:59.861022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:50:59.861022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:50:59.861022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:50:59.861022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:50:59.873323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 520/1057] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 10:50:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:50:59.874428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:50:59.969300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23cb2d-9f1f-453d-b245-39b4c4052cd9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.005560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ec6be-a66c-4e1e-8ad1-61ae17a0333d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.005768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8fe30-bebb-4655-8d3c-6acb9096b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.006843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8fe30-bebb-4655-8d3c-6acb9096b422 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.007137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8fe30-bebb-4655-8d3c-6acb9096b422 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.007454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8fe30-bebb-4655-8d3c-6acb9096b422 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.033257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.040902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915e2e4-8ac9-4865-a05c-003917c16bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.053450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e2e4-8ac9-4865-a05c-003917c16bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.053450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e2e4-8ac9-4865-a05c-003917c16bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.053450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e2e4-8ac9-4865-a05c-003917c16bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.053788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e2e4-8ac9-4865-a05c-003917c16bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.054996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e2e4-8ac9-4865-a05c-003917c16bf2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-8915e2e4-8ac9-4865-a05c-003917c16bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:51:00.072667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:00.073629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:00.074557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:51:00.074557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:51:00.074557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:00.074557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:00.076087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8fe30-bebb-4655-8d3c-6acb9096b422 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.077544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8fe30-bebb-4655-8d3c-6acb9096b422 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.078949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 521/1058] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:00.080467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab70782-884c-4247-9515-da6efeb1fdc2 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.080761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab70782-884c-4247-9515-da6efeb1fdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.081048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab70782-884c-4247-9515-da6efeb1fdc2 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.082353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 522/1059] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:00.083053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab70782-884c-4247-9515-da6efeb1fdc2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.105860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.189156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 538/1060] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:50:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:51:00.189592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf6240-2791-4f46-9ac8-eb23cc83d22d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.189826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf6240-2791-4f46-9ac8-eb23cc83d22d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.190133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf6240-2791-4f46-9ac8-eb23cc83d22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.203324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab70782-884c-4247-9515-da6efeb1fdc2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.206773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6baf6240-2791-4f46-9ac8-eb23cc83d22d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:00.208004 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6baf6240-2791-4f46-9ac8-eb23cc83d22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:00.238607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.239230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.239575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.239867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.240593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:00.240904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.242267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.253399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 523/1061] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:00.254269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.276107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 539/1062] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:00.276933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf6240-2791-4f46-9ac8-eb23cc83d22d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.298707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.336423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.357363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86ea3-d005-4d07-8a6b-8830e0db82d9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.377780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf6240-2791-4f46-9ac8-eb23cc83d22d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.389930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.390726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.391014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.391318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.402425 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.403905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.412559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.413355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.413640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.413922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.420994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:51:00.422007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:00.423444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:00.424498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:51:00.424498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:51:00.424498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:00.424498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:00.424763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.424846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 524/1063] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:00.424846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:51:00.440260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:00.441506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 540/1064] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:00.443050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.531839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.540202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ec26-18c6-40b1-817b-2e2bfec2087a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.542550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ffee4-492e-43a4-ba2d-d7b4e36d3fbc None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.560451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.573819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.574613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.574902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.575215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.577236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.577896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.578182 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.578460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.579216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:00.579531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.580896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.586444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:00.592571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 541/1065] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:00.593371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:00.593987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:00.595435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:00.596351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 525/1066] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:00.602500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.712178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8ea1b-3c55-40ee-ae6c-7e47a51d44b4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.726532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68844-ca0f-4a8a-a25c-eee80727834c None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.752714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.753395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.753680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.753958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.754811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:00.755808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.757727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.770198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.771698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.771832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.772055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.772816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:00.773148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.774522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.777564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 542/1067] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:00.783021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.784295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 526/1068] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:00.789984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.813019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.821855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:00.899342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce866f-6a52-41f5-aa71-2d4588da1927 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.909266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dd091-b610-4096-8493-b8edb3247b54 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:00.934573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.935390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.935677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.935948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.946338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:00.946957 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.947244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:00.947538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:00.947823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] PATH_INFO: `/v3/users/7c7cfeddb39e41c786556fb276fe527a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:00.948429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.961471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] RBAC: Authorizing `identity:get_user(user_id=7c7cfeddb39e41c786556fb276fe527a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:00.962904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:00.964629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 527/1069] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/users/7c7cfeddb39e41c786556fb276fe527a => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:51:00.966941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:00.968080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:00.969278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:51:00.969278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:51:00.969278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:00.969278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:00.970057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 543/1070] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:51:00.970414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:00.975863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.041157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:01.062394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:01.069861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e9c43-0b6f-4e21-81ae-f9071e5aa584 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.074132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faf841-781e-4eda-b419-12612cb6c3e4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.106215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.106898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.107217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.107497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.107683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.108506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.109153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.109431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.110151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:01.110458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.111852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.123188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 544/1071] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:01.128762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.179643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.181102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.228501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae7575-42df-4567-a663-2ebdca0ad217 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.248973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:01.261900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.262709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.262998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.263314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.269932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.272027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.283888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 528/1072] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 10:51:01.284933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:51:01.286292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:01.287375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:01.288522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 545/1073] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:51:01.293208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.303535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:01.385009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4edb-f588-4ff9-bffa-3f5b474d418f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.392741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c520c8-6037-471e-85a9-6708cc859ae5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.418474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.419368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.419662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.419942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.421198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] RBAC: Authorizing `identity:get_project(project_id=972e83f0c0d24b0f865dc9746f130093)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.422652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.424392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 529/1074] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:01 2022] GET /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:01.426105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.426760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.427110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.427425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.428204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:01.428511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.429921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.430418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.446643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 546/1075] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:01.447647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:01.472148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:01.522913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:01.534191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3ea4c-47a2-41c2-8b1d-53dfdcaa1c11 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.556977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4e743-ec39-44d2-86e0-2c440ff7417d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:01.568664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec05223-5679-4f4f-b3be-33ab915041e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.569316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec05223-5679-4f4f-b3be-33ab915041e2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.569606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec05223-5679-4f4f-b3be-33ab915041e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.569886 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec05223-5679-4f4f-b3be-33ab915041e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.570720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec05223-5679-4f4f-b3be-33ab915041e2 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=972e83f0c0d24b0f865dc9746f130093, user.id=7c7cfeddb39e41c786556fb276fe527a) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:01.571035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec05223-5679-4f4f-b3be-33ab915041e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.572485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec05223-5679-4f4f-b3be-33ab915041e2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.577818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 530/1076] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:01 2022] GET /identity/v3/role_assignments?user.id=7c7cfeddb39e41c786556fb276fe527a&scope.project.id=972e83f0c0d24b0f865dc9746f130093&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:01.590888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59302a55-57c9-4c41-9045-89b919765e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:01.591916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59302a55-57c9-4c41-9045-89b919765e2e None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:01.592206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59302a55-57c9-4c41-9045-89b919765e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:01.592486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59302a55-57c9-4c41-9045-89b919765e2e None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/54bb42878fff477099bbf56a46e9eec0/roles/3103c74eb4f5404dbb37aaba54d4aac4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:01.595101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59302a55-57c9-4c41-9045-89b919765e2e None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=54bb42878fff477099bbf56a46e9eec0, role_id=3103c74eb4f5404dbb37aaba54d4aac4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:01.596622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59302a55-57c9-4c41-9045-89b919765e2e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:01.703951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 547/1077] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:51:01 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/54bb42878fff477099bbf56a46e9eec0/roles/3103c74eb4f5404dbb37aaba54d4aac4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:51:03.588380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456efcc1-5e45-47f5-a382-a60798cbfc85 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:03.588934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456efcc1-5e45-47f5-a382-a60798cbfc85 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:03.588934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456efcc1-5e45-47f5-a382-a60798cbfc85 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:03.590319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 531/1078] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:03.596007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff17da7-5b13-4580-9a89-080efb78cbdf None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:03.596403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff17da7-5b13-4580-9a89-080efb78cbdf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:03.596717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff17da7-5b13-4580-9a89-080efb78cbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:03.613050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8ff17da7-5b13-4580-9a89-080efb78cbdf None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:03.614233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8ff17da7-5b13-4580-9a89-080efb78cbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:03.678756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 548/1079] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:03.701608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456efcc1-5e45-47f5-a382-a60798cbfc85 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:03.725650 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288978d-cc52-46e0-a9f6-8637a1e53780 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:03.725941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288978d-cc52-46e0-a9f6-8637a1e53780 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:03.726225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288978d-cc52-46e0-a9f6-8637a1e53780 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:03.727642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 549/1080] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 10:51:03.732518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de67f-a3c0-4713-98df-b1a7b3e289fd None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:03.732810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de67f-a3c0-4713-98df-b1a7b3e289fd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:03.733097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0de67f-a3c0-4713-98df-b1a7b3e289fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:03.749871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5a0de67f-a3c0-4713-98df-b1a7b3e289fd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:03.751080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5a0de67f-a3c0-4713-98df-b1a7b3e289fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:03.804681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:03.805259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456efcc1-5e45-47f5-a382-a60798cbfc85 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:03.807684 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:03.818713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 550/1081] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:51:03.844277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:03.844277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:03.844277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:03.844277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:03.844935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0de67f-a3c0-4713-98df-b1a7b3e289fd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:03.851205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:03.858723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:03.859865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:03.860733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 532/1082] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:03.866396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:03.953472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0de67f-a3c0-4713-98df-b1a7b3e289fd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:03.967962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f3d0c-3046-46f7-9071-df5705f991cc None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:03.986635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:03.987480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:03.987764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:03.988037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:03.998690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:04.001559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.002229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.002520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.002803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.003590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:04.003906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.005877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:04.007409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:04.014964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:04.015876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 551/1083] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 10:51:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:51:04.016597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 533/1084] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:04.016678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.019867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.046844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.067881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.118628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d1f9-9ef3-4c8f-a6fb-33a11cf81852 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.122368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e7a70-1243-4026-ba2c-63ee8c180e2f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.152594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.153264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.153555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.153838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.154589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:04.154902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.156321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.156714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.157578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.157865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.158144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] PATH_INFO: `/v3/users/7c7cfeddb39e41c786556fb276fe527a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.159359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] RBAC: Authorizing `identity:get_user(user_id=7c7cfeddb39e41c786556fb276fe527a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.160772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.162935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 534/1085] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/users/7c7cfeddb39e41c786556fb276fe527a => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:04.165088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 552/1086] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:04.168774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.170214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.262097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.268715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49bc2c-193e-4a8d-90f0-b340a3ff526f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.273932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a5b49-3d24-4561-80c0-2683bd2c50d4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.282116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.303381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.308277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.308277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.308277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.308277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] RBAC: Authorizing `identity:get_project(project_id=972e83f0c0d24b0f865dc9746f130093)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.308277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.309043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 535/1087] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:04.311731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.312531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.312827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.313121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.314844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.324193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.326977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:51:04.344444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:04.345799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:04.346902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:51:04.346902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:51:04.346902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:04.346902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:04.346902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 553/1088] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:51:04.352243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.412343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45ea7-1088-452b-a245-f0bdbd6a81e1 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.445532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.446425 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.446711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.447001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093/users/7c7cfeddb39e41c786556fb276fe527a/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.449002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] RBAC: Authorizing `identity:create_grant(project_id=972e83f0c0d24b0f865dc9746f130093, user_id=7c7cfeddb39e41c786556fb276fe527a, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.449715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e4f3f-435c-458c-a8a7-11ec29dcee70 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.450420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.483289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.483945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.484234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.484528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.485269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:04.485581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.486944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.499021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 554/1089] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:04.504656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.514597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 536/1090] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:51:04 2022] PUT /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093/users/7c7cfeddb39e41c786556fb276fe527a/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 10:51:04.530984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.613108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dcc7c-cfd1-4369-9527-3c74421a0271 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.649067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.649890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.650180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.650463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.658356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.660802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:51:04.676393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:04.677676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:04.678988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 555/1091] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:51:04.683970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.709314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.784804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af50d8b-8969-4a74-86e0-602259bd8569 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.818957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.819662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.819953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.820235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:04.821077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:04.821395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:04.822825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:04.840698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 537/1092] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:04.846489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:04.928207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:04.946420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eaae4-b124-48fd-b131-ebc82626db9e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:04.990300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:04.990943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:04.991253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:04.991533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:05.059131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:05.060613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:05.139415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:05.157204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 556/1093] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:05.163620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:05.244579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:05.263205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7002f8-c7b5-48fa-8603-8f1a69531890 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:05.297652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:05.298439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:05.298724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:05.299001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:05.308801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:05.310227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:51:05.327509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:05.328650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:05.329627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:51:05.329627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:51:05.329627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:51:05.329627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:05.329627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 538/1094] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:51:05.334981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:05.430931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71ef94-cfd0-4e7a-8690-458eed2c8a01 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:05.464052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:05.464709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:05.464998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:05.465280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:05.466010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:05.466323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:05.467732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:05.479829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 557/1095] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:05.485235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:05.583570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad709-3962-4538-b641-a99ea432acee None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:05.614570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:05.620043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:05.620873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:05.621163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:05.621441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:05.628428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:05.629921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:51:05.643887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:05.644853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:05.646283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 539/1096] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:51:05.651499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:05.695279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:05.749932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb7708-d83f-46dd-b88a-9dce43ea47c5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:05.783273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe07cd-7aba-4551-b575-55c271183632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:05.783934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe07cd-7aba-4551-b575-55c271183632 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:05.784225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe07cd-7aba-4551-b575-55c271183632 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:05.784509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe07cd-7aba-4551-b575-55c271183632 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:05.785298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe07cd-7aba-4551-b575-55c271183632 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:05.785614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe07cd-7aba-4551-b575-55c271183632 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:05.787138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe07cd-7aba-4551-b575-55c271183632 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:05.798872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 558/1097] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:05.805099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:05.904097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99452295-91b9-46b0-8d00-7e839ca8799d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:05.938229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a77dc-fcbd-42e8-9285-68d57017d2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:05.938881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a77dc-fcbd-42e8-9285-68d57017d2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:05.939170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a77dc-fcbd-42e8-9285-68d57017d2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:05.939480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a77dc-fcbd-42e8-9285-68d57017d2e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:05.940308 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a77dc-fcbd-42e8-9285-68d57017d2e8 None admin] RBAC: Adding query filter params (role.id=3103c74eb4f5404dbb37aaba54d4aac4, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=54bb42878fff477099bbf56a46e9eec0) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:05.940625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a77dc-fcbd-42e8-9285-68d57017d2e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:05.942050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a77dc-fcbd-42e8-9285-68d57017d2e8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:05.948432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 540/1098] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:05 2022] GET /identity/v3/role_assignments?user.id=54bb42878fff477099bbf56a46e9eec0&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=3103c74eb4f5404dbb37aaba54d4aac4 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:51:06.551005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6f02d-719d-4c94-afba-c0ead5e960d7 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.551328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6f02d-719d-4c94-afba-c0ead5e960d7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.551704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6f02d-719d-4c94-afba-c0ead5e960d7 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.553185 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 559/1099] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:06.558621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71bac6-ced7-43ba-87c5-9dc7eca5573c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.558917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71bac6-ced7-43ba-87c5-9dc7eca5573c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.559289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71bac6-ced7-43ba-87c5-9dc7eca5573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.579073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5b71bac6-ced7-43ba-87c5-9dc7eca5573c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:06.582089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5b71bac6-ced7-43ba-87c5-9dc7eca5573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:06.635959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:06.654241 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 541/1100] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:06.677782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6f02d-719d-4c94-afba-c0ead5e960d7 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:06.781690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6f02d-719d-4c94-afba-c0ead5e960d7 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:06.783220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:06.815116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:06.815972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.816444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.816761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.827787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:06.835540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:06.836583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:06.837457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 560/1101] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:06.842775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71bac6-ced7-43ba-87c5-9dc7eca5573c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:06.944193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71bac6-ced7-43ba-87c5-9dc7eca5573c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:06.977914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:06.978577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:06.978867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:06.979151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:06.979931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:06.980244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:06.981620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:06.990422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 542/1102] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:06.996170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:07.093852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7a18-c783-4c2b-99ec-3ca03200b7ba None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.132670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.133539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.133823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.134101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] PATH_INFO: `/v3/users/7c7cfeddb39e41c786556fb276fe527a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.135298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] RBAC: Authorizing `identity:get_user(user_id=7c7cfeddb39e41c786556fb276fe527a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.136736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.138350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 561/1103] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:06 2022] GET /identity/v3/users/7c7cfeddb39e41c786556fb276fe527a => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:07.144171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:07.192335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:07.241500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a575131-1bd3-4725-bf5b-840519b3a1b3 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.274290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.274928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.275236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.275517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.339773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.341217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.386862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:07.435865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 543/1104] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:07.441643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:07.472868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:07.540478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d5434-8f79-48b9-aa20-d4ee52db2494 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.573747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef87cd-b866-46ea-a067-227dac9fc686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.574589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef87cd-b866-46ea-a067-227dac9fc686 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.574874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef87cd-b866-46ea-a067-227dac9fc686 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.575151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef87cd-b866-46ea-a067-227dac9fc686 None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.576333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef87cd-b866-46ea-a067-227dac9fc686 None admin] RBAC: Authorizing `identity:get_project(project_id=972e83f0c0d24b0f865dc9746f130093)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.577746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef87cd-b866-46ea-a067-227dac9fc686 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.579437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 562/1105] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:07 2022] GET /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:07.585394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:07.680471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32d67-30a5-4ee3-b04b-5679145b8d36 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:07.712944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe1e26-2e5e-4cd4-a9f7-2e8c9436ca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:07.713584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1e26-2e5e-4cd4-a9f7-2e8c9436ca32 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:07.713870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1e26-2e5e-4cd4-a9f7-2e8c9436ca32 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:07.714149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1e26-2e5e-4cd4-a9f7-2e8c9436ca32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:07.714970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe1e26-2e5e-4cd4-a9f7-2e8c9436ca32 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=972e83f0c0d24b0f865dc9746f130093, user.id=7c7cfeddb39e41c786556fb276fe527a) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:07.715312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe1e26-2e5e-4cd4-a9f7-2e8c9436ca32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:07.716724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe1e26-2e5e-4cd4-a9f7-2e8c9436ca32 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:07.722583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 544/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:07 2022] GET /identity/v3/role_assignments?user.id=7c7cfeddb39e41c786556fb276fe527a&scope.project.id=972e83f0c0d24b0f865dc9746f130093&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:09.767115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19182a-837b-4623-8a77-7a19f5faa323 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:09.767705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19182a-837b-4623-8a77-7a19f5faa323 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:09.767705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19182a-837b-4623-8a77-7a19f5faa323 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:09.769055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 563/1107] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:09.774818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52593c38-a18b-45dc-84c7-83825c4df721 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:09.775276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52593c38-a18b-45dc-84c7-83825c4df721 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:09.775407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52593c38-a18b-45dc-84c7-83825c4df721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:09.792545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-52593c38-a18b-45dc-84c7-83825c4df721 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:09.793727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-52593c38-a18b-45dc-84c7-83825c4df721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:09.800881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:09.858345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 545/1108] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:09.883147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19182a-837b-4623-8a77-7a19f5faa323 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:09.979325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:09.983631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19182a-837b-4623-8a77-7a19f5faa323 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:10.015502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d840aa9-60d2-4459-aa8d-d9baf24192b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:10.016514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d840aa9-60d2-4459-aa8d-d9baf24192b4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:10.016970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d840aa9-60d2-4459-aa8d-d9baf24192b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:10.017615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d840aa9-60d2-4459-aa8d-d9baf24192b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:10.018778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d840aa9-60d2-4459-aa8d-d9baf24192b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:10.020954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d840aa9-60d2-4459-aa8d-d9baf24192b4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:10.022628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 564/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:10.028304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52593c38-a18b-45dc-84c7-83825c4df721 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:10.124048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52593c38-a18b-45dc-84c7-83825c4df721 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:10.157864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234b196-99fe-4358-899e-5182c13656b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:10.158535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b196-99fe-4358-899e-5182c13656b3 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:10.158827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b196-99fe-4358-899e-5182c13656b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:10.159108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b196-99fe-4358-899e-5182c13656b3 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:10.159852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234b196-99fe-4358-899e-5182c13656b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:10.161982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234b196-99fe-4358-899e-5182c13656b3 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:10.257311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:10.263132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 546/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:51:10 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:12.312365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63116ed6-0543-44fc-a48c-3207e4c369f6 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.312866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63116ed6-0543-44fc-a48c-3207e4c369f6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.312866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63116ed6-0543-44fc-a48c-3207e4c369f6 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.314195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 565/1111] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:12.320157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fa618-ad73-4f59-8053-940ec98fce1d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.320566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fa618-ad73-4f59-8053-940ec98fce1d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.320727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fa618-ad73-4f59-8053-940ec98fce1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.337849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-037fa618-ad73-4f59-8053-940ec98fce1d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:12.339066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-037fa618-ad73-4f59-8053-940ec98fce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:12.403731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 547/1112] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:12.428838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63116ed6-0543-44fc-a48c-3207e4c369f6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:12.526251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63116ed6-0543-44fc-a48c-3207e4c369f6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:12.559688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:12.560533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.560817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.561092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] PATH_INFO: `/v3/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.562609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] RBAC: Authorizing `identity:get_role(role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:12.563953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:12.565295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 566/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:12 2022] GET /identity/v3/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:12.570898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fa618-ad73-4f59-8053-940ec98fce1d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:12.666159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fa618-ad73-4f59-8053-940ec98fce1d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:12.692246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:12.699670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:12.700535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.700824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.701104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] PATH_INFO: `/v3/users/01fac881b44c465788400a25538f1755` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.713887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] RBAC: Authorizing `identity:get_user(user_id=01fac881b44c465788400a25538f1755)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:12.715375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:12.717066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 548/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:12 2022] GET /identity/v3/users/01fac881b44c465788400a25538f1755 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:12.723497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:12.757698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:12.821147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b319e17-eb05-4cb5-ae46-a30d3e7172a6 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:12.854568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:12.855224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:12.855515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:12.855792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:12.921241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:12.922697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:12.953999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:13.107003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 567/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:13.112975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:13.211121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627cd18-ef02-438f-8787-3b9a7a93200e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:13.243648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181734b-9612-46e2-8bbc-907d5728a993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:13.244493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181734b-9612-46e2-8bbc-907d5728a993 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:13.244778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181734b-9612-46e2-8bbc-907d5728a993 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:13.245057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181734b-9612-46e2-8bbc-907d5728a993 None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:13.246216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181734b-9612-46e2-8bbc-907d5728a993 None admin] RBAC: Authorizing `identity:get_project(project_id=972e83f0c0d24b0f865dc9746f130093)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:13.247691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181734b-9612-46e2-8bbc-907d5728a993 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:13.249357 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 549/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:13 2022] GET /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:13.256354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:13.356641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908105b-4945-4dd0-8bbe-8b9c1166227c None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:13.391477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b90af-f836-459b-8e3a-36dc9f0d8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:13.392071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b90af-f836-459b-8e3a-36dc9f0d8a52 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:13.392361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b90af-f836-459b-8e3a-36dc9f0d8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:13.392665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b90af-f836-459b-8e3a-36dc9f0d8a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:13.393496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b90af-f836-459b-8e3a-36dc9f0d8a52 None admin] RBAC: Adding query filter params (role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=972e83f0c0d24b0f865dc9746f130093, user.id=01fac881b44c465788400a25538f1755) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:13.393814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b90af-f836-459b-8e3a-36dc9f0d8a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:13.396296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b90af-f836-459b-8e3a-36dc9f0d8a52 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:13.398118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:13.402095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 568/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:13 2022] GET /identity/v3/role_assignments?user.id=01fac881b44c465788400a25538f1755&scope.project.id=972e83f0c0d24b0f865dc9746f130093&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:15.400077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fe59f-91df-4a62-ad6a-8d8cb0711d4d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.400631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fe59f-91df-4a62-ad6a-8d8cb0711d4d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.400631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fe59f-91df-4a62-ad6a-8d8cb0711d4d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.402025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 550/1118] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:15.407209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7e5ff-f069-4c0d-81a8-263c449a7e7c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.407580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7e5ff-f069-4c0d-81a8-263c449a7e7c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.407779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7e5ff-f069-4c0d-81a8-263c449a7e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.424899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0dc7e5ff-f069-4c0d-81a8-263c449a7e7c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:15.426098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0dc7e5ff-f069-4c0d-81a8-263c449a7e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:15.493427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 569/1119] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:15.516077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fe59f-91df-4a62-ad6a-8d8cb0711d4d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:15.571672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:15.616660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fe59f-91df-4a62-ad6a-8d8cb0711d4d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:15.649617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:15.650515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.650761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.651044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] PATH_INFO: `/v3/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.652545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] RBAC: Authorizing `identity:get_role(role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:15.653912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:15.655292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 551/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:15 2022] GET /identity/v3/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:15.661556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7e5ff-f069-4c0d-81a8-263c449a7e7c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:15.756625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7e5ff-f069-4c0d-81a8-263c449a7e7c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:15.782266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:15.790364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:15.791266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.791607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.791884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] PATH_INFO: `/v3/users/01fac881b44c465788400a25538f1755` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.795732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] RBAC: Authorizing `identity:get_user(user_id=01fac881b44c465788400a25538f1755)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:15.795732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:15.796235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 570/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:15 2022] GET /identity/v3/users/01fac881b44c465788400a25538f1755 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:15.802998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:15.901585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74ec6-50ea-4da8-a540-fef0507208bf None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:15.924088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:15.936096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151e333-b455-4d6b-b374-e4aa1d16870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:15.936957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151e333-b455-4d6b-b374-e4aa1d16870f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:15.937290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151e333-b455-4d6b-b374-e4aa1d16870f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:15.937534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151e333-b455-4d6b-b374-e4aa1d16870f None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:15.938719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151e333-b455-4d6b-b374-e4aa1d16870f None admin] RBAC: Authorizing `identity:get_project(project_id=972e83f0c0d24b0f865dc9746f130093)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:15.940205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151e333-b455-4d6b-b374-e4aa1d16870f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:15.942733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 552/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:15 2022] GET /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:15.948709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:16.043353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeeb04-aa46-4f48-8faa-2ebc3faf389f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:16.076342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3b998-4d09-4f6b-847c-021724df6703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:16.077211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3b998-4d09-4f6b-847c-021724df6703 None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:16.077496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3b998-4d09-4f6b-847c-021724df6703 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:16.077774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3b998-4d09-4f6b-847c-021724df6703 None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093/users/01fac881b44c465788400a25538f1755/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:16.079948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3b998-4d09-4f6b-847c-021724df6703 None admin] RBAC: Authorizing `identity:create_grant(project_id=972e83f0c0d24b0f865dc9746f130093, user_id=01fac881b44c465788400a25538f1755, role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:16.081378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3b998-4d09-4f6b-847c-021724df6703 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:16.147164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 571/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:51:15 2022] PUT /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093/users/01fac881b44c465788400a25538f1755/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:51:18.148604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810ddd-d51f-41d2-8609-979df071dd8c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.149162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810ddd-d51f-41d2-8609-979df071dd8c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.149162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810ddd-d51f-41d2-8609-979df071dd8c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.150518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 553/1124] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:18.155745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735872f8-fc61-44bd-8f8b-e48a3e0ce10f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.156131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735872f8-fc61-44bd-8f8b-e48a3e0ce10f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.156314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735872f8-fc61-44bd-8f8b-e48a3e0ce10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.173473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-735872f8-fc61-44bd-8f8b-e48a3e0ce10f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:18.174588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-735872f8-fc61-44bd-8f8b-e48a3e0ce10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:18.223521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:18.239195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 572/1125] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:18.262864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810ddd-d51f-41d2-8609-979df071dd8c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:18.366449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810ddd-d51f-41d2-8609-979df071dd8c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:18.399074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:18.399956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.400243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.400520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] PATH_INFO: `/v3/roles/bcf717d7373e4ca3a00de9cea81604f0` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.401962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] RBAC: Authorizing `identity:get_role(role_id=bcf717d7373e4ca3a00de9cea81604f0)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:18.403342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:18.404702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 554/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:18 2022] GET /identity/v3/roles/bcf717d7373e4ca3a00de9cea81604f0 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:18.411155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735872f8-fc61-44bd-8f8b-e48a3e0ce10f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:18.507768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735872f8-fc61-44bd-8f8b-e48a3e0ce10f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:18.540880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:18.541740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.542028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.542309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] PATH_INFO: `/v3/users/01fac881b44c465788400a25538f1755` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.543532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] RBAC: Authorizing `identity:get_user(user_id=01fac881b44c465788400a25538f1755)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:18.544972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:18.546586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 573/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:18 2022] GET /identity/v3/users/01fac881b44c465788400a25538f1755 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:18.552346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:18.613124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:18.654664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f22af4-51eb-4419-8d6a-f6572665f6d0 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:18.686744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:18.687406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.687690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.687968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.752861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:18.754880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:18.808672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:18.848780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 555/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:18.854782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:18.908693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:18.952596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4080b6-7c00-4798-b078-b20b2feff301 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:18.985838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d423054-6e18-4942-ba10-ec57d98d6dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:18.986684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423054-6e18-4942-ba10-ec57d98d6dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:18.986970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423054-6e18-4942-ba10-ec57d98d6dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:18.987284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423054-6e18-4942-ba10-ec57d98d6dd9 None admin] PATH_INFO: `/v3/projects/972e83f0c0d24b0f865dc9746f130093` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:18.989038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d423054-6e18-4942-ba10-ec57d98d6dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=972e83f0c0d24b0f865dc9746f130093)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:18.990477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d423054-6e18-4942-ba10-ec57d98d6dd9 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:18.992181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 574/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:18 2022] GET /identity/v3/projects/972e83f0c0d24b0f865dc9746f130093 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:18.998208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:19.094615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873ee52-793f-42cc-b344-ffe8b923bf2f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:19.127611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0a647-a936-4d4b-b788-89fad6dfec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:19.128257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0a647-a936-4d4b-b788-89fad6dfec38 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:19.128545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0a647-a936-4d4b-b788-89fad6dfec38 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:19.128828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0a647-a936-4d4b-b788-89fad6dfec38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:19.129928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0a647-a936-4d4b-b788-89fad6dfec38 None admin] RBAC: Adding query filter params (role.id=bcf717d7373e4ca3a00de9cea81604f0, scope.project.id=972e83f0c0d24b0f865dc9746f130093, user.id=01fac881b44c465788400a25538f1755) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:19.130249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0a647-a936-4d4b-b788-89fad6dfec38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:19.131700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0a647-a936-4d4b-b788-89fad6dfec38 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:19.137338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 556/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:18 2022] GET /identity/v3/role_assignments?user.id=01fac881b44c465788400a25538f1755&scope.project.id=972e83f0c0d24b0f865dc9746f130093&role.id=bcf717d7373e4ca3a00de9cea81604f0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:21.185241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02439cf-ab6e-40e7-9c3b-e47199d8183f None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:21.185785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02439cf-ab6e-40e7-9c3b-e47199d8183f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:21.185785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02439cf-ab6e-40e7-9c3b-e47199d8183f None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:21.187067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 575/1131] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:21.192722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f565d5-8d3e-4317-b4b5-e1fa4a546ad1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:21.193111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f565d5-8d3e-4317-b4b5-e1fa4a546ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:21.193287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f565d5-8d3e-4317-b4b5-e1fa4a546ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:21.209767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a9f565d5-8d3e-4317-b4b5-e1fa4a546ad1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:21.210958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a9f565d5-8d3e-4317-b4b5-e1fa4a546ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:21.227414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:21.277127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 557/1132] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:51:21.299705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02439cf-ab6e-40e7-9c3b-e47199d8183f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:21.401007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02439cf-ab6e-40e7-9c3b-e47199d8183f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:21.423000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:21.434842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15db2c-4e9e-4e19-b79c-6da24f4235f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:21.437903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15db2c-4e9e-4e19-b79c-6da24f4235f1 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:21.437903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15db2c-4e9e-4e19-b79c-6da24f4235f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:21.437903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15db2c-4e9e-4e19-b79c-6da24f4235f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:21.437903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15db2c-4e9e-4e19-b79c-6da24f4235f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:21.438792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15db2c-4e9e-4e19-b79c-6da24f4235f1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:21.440424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 576/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:21.446142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f565d5-8d3e-4317-b4b5-e1fa4a546ad1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:21.542554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f565d5-8d3e-4317-b4b5-e1fa4a546ad1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:21.574889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d421a9-9f96-414b-9415-945b18140eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:21.575558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d421a9-9f96-414b-9415-945b18140eba None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:21.575843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d421a9-9f96-414b-9415-945b18140eba None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:21.576123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d421a9-9f96-414b-9415-945b18140eba None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:21.576829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d421a9-9f96-414b-9415-945b18140eba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:21.578225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d421a9-9f96-414b-9415-945b18140eba None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:21.674569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 558/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:51:21 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:51:23.802908 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07536d-0395-4f34-8837-3e89c07662dd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:23.803484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07536d-0395-4f34-8837-3e89c07662dd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:23.803484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07536d-0395-4f34-8837-3e89c07662dd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:23.804867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 577/1135] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:23.810077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da445ff-6b48-4106-8a6a-7c5615830c96 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:23.810473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da445ff-6b48-4106-8a6a-7c5615830c96 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:23.810645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da445ff-6b48-4106-8a6a-7c5615830c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:23.817143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:23.827699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7da445ff-6b48-4106-8a6a-7c5615830c96 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:23.828883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7da445ff-6b48-4106-8a6a-7c5615830c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:23.894064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 559/1136] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:23.919476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07536d-0395-4f34-8837-3e89c07662dd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:24.021020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07536d-0395-4f34-8837-3e89c07662dd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:24.054376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e1736-c49b-46b9-9a26-92a5ca34af16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:24.055098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e1736-c49b-46b9-9a26-92a5ca34af16 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:24.055412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e1736-c49b-46b9-9a26-92a5ca34af16 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:24.055693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e1736-c49b-46b9-9a26-92a5ca34af16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:24.056864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e1736-c49b-46b9-9a26-92a5ca34af16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:24.058298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e1736-c49b-46b9-9a26-92a5ca34af16 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:24.059940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 578/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:51:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:24.065815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da445ff-6b48-4106-8a6a-7c5615830c96 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:24.162550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da445ff-6b48-4106-8a6a-7c5615830c96 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:24.196591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd841d8d-1ae6-4d68-9cfe-43e113f70336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:24.197247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd841d8d-1ae6-4d68-9cfe-43e113f70336 None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:24.197532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd841d8d-1ae6-4d68-9cfe-43e113f70336 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:24.197978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd841d8d-1ae6-4d68-9cfe-43e113f70336 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:24.198691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd841d8d-1ae6-4d68-9cfe-43e113f70336 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:24.200128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd841d8d-1ae6-4d68-9cfe-43e113f70336 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:24.206552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:24.376139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 560/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:51:24 2022] POST /identity/v3/users => generated 286 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 10:51:26.431331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b345c-3154-4b3e-881e-6620fd9c87f3 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.431894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b345c-3154-4b3e-881e-6620fd9c87f3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.431894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b345c-3154-4b3e-881e-6620fd9c87f3 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.433248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 579/1139] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:26.438487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793de727-6b29-4583-95e2-b8df8e384c0b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.438914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793de727-6b29-4583-95e2-b8df8e384c0b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.439050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793de727-6b29-4583-95e2-b8df8e384c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.456101 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-793de727-6b29-4583-95e2-b8df8e384c0b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:26.457459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-793de727-6b29-4583-95e2-b8df8e384c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:26.520890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 561/1140] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:26.543673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b345c-3154-4b3e-881e-6620fd9c87f3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:26.601580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:26.641644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b345c-3154-4b3e-881e-6620fd9c87f3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:26.674110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:26.674884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.675167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.676067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.685898 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:26.693430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:26.694584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:26.695924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 580/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:26.700386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793de727-6b29-4583-95e2-b8df8e384c0b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:26.796072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793de727-6b29-4583-95e2-b8df8e384c0b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:26.815456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:26.828675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:26.829330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.829613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.829889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:26.830660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:26.833063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:26.833063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:26.840194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 562/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:26.846179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:26.946682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c37d0b-6fac-4e87-a1a4-cf8c6eb26a20 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:26.963191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:26.980619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:26.981480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:26.981765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:26.982040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] PATH_INFO: `/v3/users/cc374aa87bfe441ea54783d0a63decc0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:27.007855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cc374aa87bfe441ea54783d0a63decc0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:27.009361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:27.011011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 581/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:26 2022] GET /identity/v3/users/cc374aa87bfe441ea54783d0a63decc0 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:27.016862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:27.114952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8fbe-423d-47ee-95da-9d0a44b18846 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:27.147407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:27.148044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:27.148330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:27.148607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:27.195711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:27.213729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:27.215253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:27.310383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 563/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:27.316213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:27.413243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4890607-734d-4dc1-ae9f-b5181e5445e5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:27.446324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030beac-a27c-4b2b-a2a8-4626f0a127a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:27.447209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030beac-a27c-4b2b-a2a8-4626f0a127a4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:27.447490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030beac-a27c-4b2b-a2a8-4626f0a127a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:27.447770 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030beac-a27c-4b2b-a2a8-4626f0a127a4 None admin] PATH_INFO: `/v3/projects/d8097cf0e1e0415781fab83830b64bdd` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:27.448939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030beac-a27c-4b2b-a2a8-4626f0a127a4 None admin] RBAC: Authorizing `identity:get_project(project_id=d8097cf0e1e0415781fab83830b64bdd)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:27.450373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030beac-a27c-4b2b-a2a8-4626f0a127a4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:27.452281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 582/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:27 2022] GET /identity/v3/projects/d8097cf0e1e0415781fab83830b64bdd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:27.458718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:27.536236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:27.553103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e64dc7-bea2-4845-a1ef-1c924e157069 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:27.587251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff240815-7896-4498-ba1c-e350bfad44f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:27.587900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240815-7896-4498-ba1c-e350bfad44f1 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:27.588187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240815-7896-4498-ba1c-e350bfad44f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:27.588496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240815-7896-4498-ba1c-e350bfad44f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:27.589323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff240815-7896-4498-ba1c-e350bfad44f1 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=d8097cf0e1e0415781fab83830b64bdd, user.id=cc374aa87bfe441ea54783d0a63decc0) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:27.589639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff240815-7896-4498-ba1c-e350bfad44f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:27.591064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff240815-7896-4498-ba1c-e350bfad44f1 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:27.596639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 564/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:27 2022] GET /identity/v3/role_assignments?user.id=cc374aa87bfe441ea54783d0a63decc0&scope.project.id=d8097cf0e1e0415781fab83830b64bdd&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:29.600438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de900833-820e-4cc8-aad8-fa7486c0b591 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:29.601003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de900833-820e-4cc8-aad8-fa7486c0b591 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:29.601003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de900833-820e-4cc8-aad8-fa7486c0b591 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:29.602345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 583/1147] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:29.607514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b246f-3416-4f21-95a6-7aa496a8e4b6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:29.607914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b246f-3416-4f21-95a6-7aa496a8e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:29.608091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b246f-3416-4f21-95a6-7aa496a8e4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:29.624835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-204b246f-3416-4f21-95a6-7aa496a8e4b6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:29.626007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-204b246f-3416-4f21-95a6-7aa496a8e4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:29.690132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 565/1148] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:29.713717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de900833-820e-4cc8-aad8-fa7486c0b591 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:29.760113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:29.812655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de900833-820e-4cc8-aad8-fa7486c0b591 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:29.846098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:29.846870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:29.847210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:29.847493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:29.857259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:29.864927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:29.866057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:29.867133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 584/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:29.872061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b246f-3416-4f21-95a6-7aa496a8e4b6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:29.945993 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:29.967633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b246f-3416-4f21-95a6-7aa496a8e4b6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:29.999859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:30.000515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:30.000801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:30.001078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:30.001845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:30.002159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:30.004097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:30.012084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 566/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:30.017940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:30.121503 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bbe25-f6bd-49d6-9f41-c52ea9fda18b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:30.123722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:30.154159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:30.155016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:30.155327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:30.155602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] PATH_INFO: `/v3/users/cc374aa87bfe441ea54783d0a63decc0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:30.156770 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] RBAC: Authorizing `identity:get_user(user_id=cc374aa87bfe441ea54783d0a63decc0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:30.158536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:30.160605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 585/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:30 2022] GET /identity/v3/users/cc374aa87bfe441ea54783d0a63decc0 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:30.166349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:30.264242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f615cf9-983a-413a-9f12-2a9e04d6bf60 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:30.296135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eff67e-2d4c-4e36-86e2-291b690ef545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:30.296989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eff67e-2d4c-4e36-86e2-291b690ef545 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:30.297272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eff67e-2d4c-4e36-86e2-291b690ef545 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:30.297549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eff67e-2d4c-4e36-86e2-291b690ef545 None admin] PATH_INFO: `/v3/projects/d8097cf0e1e0415781fab83830b64bdd` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:30.298711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eff67e-2d4c-4e36-86e2-291b690ef545 None admin] RBAC: Authorizing `identity:get_project(project_id=d8097cf0e1e0415781fab83830b64bdd)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:30.300736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eff67e-2d4c-4e36-86e2-291b690ef545 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:30.302463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 567/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:30 2022] GET /identity/v3/projects/d8097cf0e1e0415781fab83830b64bdd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:30.308223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:30.405277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f96d8b-8e8c-47a7-b1ea-dedcad413a42 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:30.440014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406c28f-91e8-48f7-9196-fdb07a8b690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:30.440948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406c28f-91e8-48f7-9196-fdb07a8b690c None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:30.441358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406c28f-91e8-48f7-9196-fdb07a8b690c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:30.441481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406c28f-91e8-48f7-9196-fdb07a8b690c None admin] PATH_INFO: `/v3/projects/d8097cf0e1e0415781fab83830b64bdd/users/cc374aa87bfe441ea54783d0a63decc0/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:30.443455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406c28f-91e8-48f7-9196-fdb07a8b690c None admin] RBAC: Authorizing `identity:create_grant(project_id=d8097cf0e1e0415781fab83830b64bdd, user_id=cc374aa87bfe441ea54783d0a63decc0, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:30.444882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406c28f-91e8-48f7-9196-fdb07a8b690c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:30.512750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 586/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:51:30 2022] PUT /identity/v3/projects/d8097cf0e1e0415781fab83830b64bdd/users/cc374aa87bfe441ea54783d0a63decc0/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:51:32.545958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c250-940a-44dc-92aa-0b2c40dbcb37 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:32.546576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c250-940a-44dc-92aa-0b2c40dbcb37 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:32.546576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754c250-940a-44dc-92aa-0b2c40dbcb37 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:32.547914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 568/1154] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:32.553268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f55b0-f006-429b-b2c1-e08303b6868e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:32.553670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f55b0-f006-429b-b2c1-e08303b6868e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:32.553832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f55b0-f006-429b-b2c1-e08303b6868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:32.570368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-107f55b0-f006-429b-b2c1-e08303b6868e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:32.571628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-107f55b0-f006-429b-b2c1-e08303b6868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:32.597950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:32.637402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 587/1155] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:32.661970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754c250-940a-44dc-92aa-0b2c40dbcb37 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:32.671302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:32.761178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754c250-940a-44dc-92aa-0b2c40dbcb37 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:32.793489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:32.794268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:32.794552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:32.794829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:32.804607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:32.812063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:32.813323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:32.814447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 569/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:32.819757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f55b0-f006-429b-b2c1-e08303b6868e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:32.916827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f55b0-f006-429b-b2c1-e08303b6868e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:32.950978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:32.951678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:32.951970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:32.952254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:32.953009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:32.953327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:32.954702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:32.963293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 588/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:32.969191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:33.035418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:33.069025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddb753-8a85-4ab5-83e4-8567dbe61605 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:33.101692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:33.102552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:33.102836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:33.103113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] PATH_INFO: `/v3/users/cc374aa87bfe441ea54783d0a63decc0` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:33.104300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] RBAC: Authorizing `identity:get_user(user_id=cc374aa87bfe441ea54783d0a63decc0)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:33.105715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:33.107333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 570/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:32 2022] GET /identity/v3/users/cc374aa87bfe441ea54783d0a63decc0 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:33.113603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:33.139429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:33.213226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0dd87-5da1-48bf-a3e6-aabfefea5c67 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:33.246551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:33.247223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:33.247518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:33.247801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:33.312229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:33.313658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:33.336875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:33.408420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 589/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:33.414859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:33.512033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26143a9c-ab4c-4c2b-b5d1-c7c5bd980dc4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:33.544907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b648f61-d521-48d5-a3d8-34fdde36ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:33.545757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b648f61-d521-48d5-a3d8-34fdde36ce84 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:33.546046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b648f61-d521-48d5-a3d8-34fdde36ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:33.546589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b648f61-d521-48d5-a3d8-34fdde36ce84 None admin] PATH_INFO: `/v3/projects/d8097cf0e1e0415781fab83830b64bdd` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:33.547520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b648f61-d521-48d5-a3d8-34fdde36ce84 None admin] RBAC: Authorizing `identity:get_project(project_id=d8097cf0e1e0415781fab83830b64bdd)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:33.548943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b648f61-d521-48d5-a3d8-34fdde36ce84 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:33.550746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 571/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:33 2022] GET /identity/v3/projects/d8097cf0e1e0415781fab83830b64bdd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:33.556732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:33.654561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1b526-482f-46f0-bb03-a3689ca0ef32 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:33.678823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:33.689034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f00c22-fd1c-4af2-a8cd-483fc5442499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:33.690122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f00c22-fd1c-4af2-a8cd-483fc5442499 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:33.690122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f00c22-fd1c-4af2-a8cd-483fc5442499 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:33.690317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f00c22-fd1c-4af2-a8cd-483fc5442499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:33.691725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f00c22-fd1c-4af2-a8cd-483fc5442499 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=d8097cf0e1e0415781fab83830b64bdd, user.id=cc374aa87bfe441ea54783d0a63decc0) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:33.692046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f00c22-fd1c-4af2-a8cd-483fc5442499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:33.693480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f00c22-fd1c-4af2-a8cd-483fc5442499 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:33.700850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 590/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:33 2022] GET /identity/v3/role_assignments?user.id=cc374aa87bfe441ea54783d0a63decc0&scope.project.id=d8097cf0e1e0415781fab83830b64bdd&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:35.745869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acef466-4305-4ee3-a714-2b1326f231df None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:35.746434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acef466-4305-4ee3-a714-2b1326f231df None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:35.746434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acef466-4305-4ee3-a714-2b1326f231df None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:35.747858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 572/1162] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:35.753194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a235-581f-44ce-ac52-54effe8d2300 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:35.753607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a235-581f-44ce-ac52-54effe8d2300 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:35.753756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992a235-581f-44ce-ac52-54effe8d2300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:35.770190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0992a235-581f-44ce-ac52-54effe8d2300 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:35.771398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0992a235-581f-44ce-ac52-54effe8d2300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:35.836325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 591/1163] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:35.859143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acef466-4305-4ee3-a714-2b1326f231df None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:35.862048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:35.959559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acef466-4305-4ee3-a714-2b1326f231df None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:35.993497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:35.994137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:35.994426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:35.994707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:36.059935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:36.061317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:36.063062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:36.156248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 573/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:36.162407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992a235-581f-44ce-ac52-54effe8d2300 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:36.258899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992a235-581f-44ce-ac52-54effe8d2300 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:36.275380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:36.291523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a60528-1ecf-4f45-bc8a-b229b52307f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:36.292315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60528-1ecf-4f45-bc8a-b229b52307f2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:36.292600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60528-1ecf-4f45-bc8a-b229b52307f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:36.292876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60528-1ecf-4f45-bc8a-b229b52307f2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:36.299591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a60528-1ecf-4f45-bc8a-b229b52307f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:36.301920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a60528-1ecf-4f45-bc8a-b229b52307f2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-78a60528-1ecf-4f45-bc8a-b229b52307f2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:36.317262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:36.318538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:36.319886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 592/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:51:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:36.324375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:36.420634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80924f92-9c7a-489b-9aa7-adce916b35b7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:36.422154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:36.453690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78c46e-0e80-4bdc-bb45-9405c6df8ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:36.454359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c46e-0e80-4bdc-bb45-9405c6df8ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:36.454839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c46e-0e80-4bdc-bb45-9405c6df8ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:36.455161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c46e-0e80-4bdc-bb45-9405c6df8ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:36.456164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c46e-0e80-4bdc-bb45-9405c6df8ff2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:36.456700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c46e-0e80-4bdc-bb45-9405c6df8ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:36.458340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c46e-0e80-4bdc-bb45-9405c6df8ff2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:36.465375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 574/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 10:51:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:38.476631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14f7f2-b05f-43f5-ac93-ff984ed643bb None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:38.477189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14f7f2-b05f-43f5-ac93-ff984ed643bb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:38.477189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14f7f2-b05f-43f5-ac93-ff984ed643bb None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:38.478573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 593/1167] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:38.484125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863aca1a-e7ee-4bfb-9212-79e55bd70137 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:38.484552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863aca1a-e7ee-4bfb-9212-79e55bd70137 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:38.484694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863aca1a-e7ee-4bfb-9212-79e55bd70137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:38.503639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-863aca1a-e7ee-4bfb-9212-79e55bd70137 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:38.504832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-863aca1a-e7ee-4bfb-9212-79e55bd70137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:38.569300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 575/1168] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:38.593949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14f7f2-b05f-43f5-ac93-ff984ed643bb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:38.693096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14f7f2-b05f-43f5-ac93-ff984ed643bb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:38.733112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9725-88a4-4f20-a5f9-27c22678f074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:38.733692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9725-88a4-4f20-a5f9-27c22678f074 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:38.733975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9725-88a4-4f20-a5f9-27c22678f074 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:38.734252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9725-88a4-4f20-a5f9-27c22678f074 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:38.734990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9725-88a4-4f20-a5f9-27c22678f074 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:38.736386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9725-88a4-4f20-a5f9-27c22678f074 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:38.810329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 594/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Thu Jun 23 10:51:38 2022] POST /identity/v3/domains => generated 267 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:40.852227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56261474-00d7-4ef1-9ac4-34af452d6ce9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:40.852788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56261474-00d7-4ef1-9ac4-34af452d6ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:40.852788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56261474-00d7-4ef1-9ac4-34af452d6ce9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:40.854153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 576/1170] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:40.859236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ac457-9c36-4a0f-8561-182c399d4f3d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:40.859662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ac457-9c36-4a0f-8561-182c399d4f3d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:40.859814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ac457-9c36-4a0f-8561-182c399d4f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:40.877017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-003ac457-9c36-4a0f-8561-182c399d4f3d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:40.879036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-003ac457-9c36-4a0f-8561-182c399d4f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:40.907568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:40.945706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 595/1171] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:40.969081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56261474-00d7-4ef1-9ac4-34af452d6ce9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:41.061759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:41.068897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56261474-00d7-4ef1-9ac4-34af452d6ce9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:41.102391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a724eb-db2e-4463-b157-31eb2a61f828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:41.103209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a724eb-db2e-4463-b157-31eb2a61f828 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:41.103504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a724eb-db2e-4463-b157-31eb2a61f828 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:41.103781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a724eb-db2e-4463-b157-31eb2a61f828 None admin] PATH_INFO: `/v3/domains/4ca81f29126d452bbc1fab9448020fc2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:41.114129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a724eb-db2e-4463-b157-31eb2a61f828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4ca81f29126d452bbc1fab9448020fc2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:41.115668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a724eb-db2e-4463-b157-31eb2a61f828 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:41.117283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 577/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Thu Jun 23 10:51:40 2022] GET /identity/v3/domains/4ca81f29126d452bbc1fab9448020fc2 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:41.123270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ac457-9c36-4a0f-8561-182c399d4f3d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:41.220367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ac457-9c36-4a0f-8561-182c399d4f3d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:41.253266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0469c-e4b2-4e93-9931-4013e95be1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:41.253913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0469c-e4b2-4e93-9931-4013e95be1d6 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:41.254203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0469c-e4b2-4e93-9931-4013e95be1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:41.254485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0469c-e4b2-4e93-9931-4013e95be1d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:41.255211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0469c-e4b2-4e93-9931-4013e95be1d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:41.256643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0469c-e4b2-4e93-9931-4013e95be1d6 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:41.341080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 596/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:51:41 2022] POST /identity/v3/projects => generated 361 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:43.374964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c06c2-8fcd-4ffb-beb6-9347fc6d15ea None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:43.375556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c06c2-8fcd-4ffb-beb6-9347fc6d15ea None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:43.375556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c06c2-8fcd-4ffb-beb6-9347fc6d15ea None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:43.376931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 578/1174] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:43.382521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d27e-f544-4f6a-93b9-43eec8df82ea None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:43.382949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d27e-f544-4f6a-93b9-43eec8df82ea None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:43.383078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346d27e-f544-4f6a-93b9-43eec8df82ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:43.399973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a346d27e-f544-4f6a-93b9-43eec8df82ea None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:43.401143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a346d27e-f544-4f6a-93b9-43eec8df82ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:43.414816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:43.469116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 597/1175] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:43.493651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c06c2-8fcd-4ffb-beb6-9347fc6d15ea None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:43.599666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c06c2-8fcd-4ffb-beb6-9347fc6d15ea None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:43.632853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7d914-201d-456b-8d78-2309777f752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:43.633645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d914-201d-456b-8d78-2309777f752c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:43.633929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d914-201d-456b-8d78-2309777f752c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:43.634212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d914-201d-456b-8d78-2309777f752c None admin] PATH_INFO: `/v3/domains/4ca81f29126d452bbc1fab9448020fc2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:43.635397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7d914-201d-456b-8d78-2309777f752c None admin] RBAC: Authorizing `identity:get_domain(domain_id=4ca81f29126d452bbc1fab9448020fc2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:43.636825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7d914-201d-456b-8d78-2309777f752c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:43.638411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 579/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Thu Jun 23 10:51:43 2022] GET /identity/v3/domains/4ca81f29126d452bbc1fab9448020fc2 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:43.644076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346d27e-f544-4f6a-93b9-43eec8df82ea None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:43.738958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346d27e-f544-4f6a-93b9-43eec8df82ea None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:43.771604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73616726-2b74-4df9-9710-8a90a30a34bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:43.772265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73616726-2b74-4df9-9710-8a90a30a34bf None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:43.772547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73616726-2b74-4df9-9710-8a90a30a34bf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:43.772821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73616726-2b74-4df9-9710-8a90a30a34bf None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:43.773517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73616726-2b74-4df9-9710-8a90a30a34bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:43.774933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73616726-2b74-4df9-9710-8a90a30a34bf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:43.848828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 598/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:51:43 2022] POST /identity/v3/users => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:51:45.911936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7e86-fe39-4d5d-b55b-71ba3dd7843a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:45.912459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7e86-fe39-4d5d-b55b-71ba3dd7843a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:45.912548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7e86-fe39-4d5d-b55b-71ba3dd7843a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:45.913864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 580/1178] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:45.919317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e4562-d87e-4e58-af1d-29b3dd26a792 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:45.919752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e4562-d87e-4e58-af1d-29b3dd26a792 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:45.919878 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e4562-d87e-4e58-af1d-29b3dd26a792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:45.931357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:45.936578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5f1e4562-d87e-4e58-af1d-29b3dd26a792 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:45.937755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5f1e4562-d87e-4e58-af1d-29b3dd26a792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:46.001217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 599/1179] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:46.025075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be7e86-fe39-4d5d-b55b-71ba3dd7843a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:46.050364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:46.129222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be7e86-fe39-4d5d-b55b-71ba3dd7843a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:46.165419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:46.170604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:46.170903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:46.173350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:46.183454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:46.191841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:46.192965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:46.194052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 581/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:46.198968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e4562-d87e-4e58-af1d-29b3dd26a792 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:46.294052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e4562-d87e-4e58-af1d-29b3dd26a792 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:46.327600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:46.328250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:46.328531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:46.328805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:46.329542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:46.329846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:46.331942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:46.334052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:46.340099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 600/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:46.346139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:46.431727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:46.447358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988a6a5-7614-429b-84cd-39275b32232f None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:46.487332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:46.488188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:46.488478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:46.488759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] PATH_INFO: `/v3/users/39a726ead61541a2af734994c25444f2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:46.501364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] RBAC: Authorizing `identity:get_user(user_id=39a726ead61541a2af734994c25444f2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:46.503597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:46.505240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 582/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:46 2022] GET /identity/v3/users/39a726ead61541a2af734994c25444f2 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:46.510938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:46.608623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e4d1e-fdb1-4ea9-b568-c4dab880a781 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:46.641566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:46.642206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:46.642492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:46.642822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:46.704765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:46.708464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:46.709923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:46.815856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 601/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:46.821719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:46.917626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee5bc5-3095-4f45-ae4e-b8ceb9c82684 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:46.950605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4edd-8f8a-45c4-9546-c930c98a9313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:46.951476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4edd-8f8a-45c4-9546-c930c98a9313 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:46.951760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4edd-8f8a-45c4-9546-c930c98a9313 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:46.952034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4edd-8f8a-45c4-9546-c930c98a9313 None admin] PATH_INFO: `/v3/projects/07756f54a9f4427ca491d06dfbccbc0d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:46.953184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4edd-8f8a-45c4-9546-c930c98a9313 None admin] RBAC: Authorizing `identity:get_project(project_id=07756f54a9f4427ca491d06dfbccbc0d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:46.954606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4edd-8f8a-45c4-9546-c930c98a9313 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:46.956431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 583/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:46 2022] GET /identity/v3/projects/07756f54a9f4427ca491d06dfbccbc0d => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:46.962377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:47.055929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:47.059534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5ed6e-371a-48d7-bf71-076f67ce0f3d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:47.093259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f671c1-60c4-47ed-ac6b-aef0bfc83093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:47.093904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f671c1-60c4-47ed-ac6b-aef0bfc83093 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:47.094190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f671c1-60c4-47ed-ac6b-aef0bfc83093 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:47.094471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f671c1-60c4-47ed-ac6b-aef0bfc83093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:47.095326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f671c1-60c4-47ed-ac6b-aef0bfc83093 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=07756f54a9f4427ca491d06dfbccbc0d, user.id=39a726ead61541a2af734994c25444f2) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:47.095642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f671c1-60c4-47ed-ac6b-aef0bfc83093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:47.097138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f671c1-60c4-47ed-ac6b-aef0bfc83093 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:47.102391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 602/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:46 2022] GET /identity/v3/role_assignments?user.id=39a726ead61541a2af734994c25444f2&scope.project.id=07756f54a9f4427ca491d06dfbccbc0d&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:49.098716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf530d-a0e3-47d8-9d03-9afad3136d60 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:49.099273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf530d-a0e3-47d8-9d03-9afad3136d60 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:49.099381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf530d-a0e3-47d8-9d03-9afad3136d60 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:49.100653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 584/1186] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:49.105814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff32c0b-60ef-4c5d-b2e7-0d4b8dd055d2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:49.106227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff32c0b-60ef-4c5d-b2e7-0d4b8dd055d2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:49.106368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff32c0b-60ef-4c5d-b2e7-0d4b8dd055d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:49.127984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8ff32c0b-60ef-4c5d-b2e7-0d4b8dd055d2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:49.129164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8ff32c0b-60ef-4c5d-b2e7-0d4b8dd055d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:49.196441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 603/1187] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:49.219156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf530d-a0e3-47d8-9d03-9afad3136d60 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:49.231334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:49.317632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf530d-a0e3-47d8-9d03-9afad3136d60 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:49.350307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:49.351080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:49.351391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:49.351669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:49.361307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:49.368965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:49.369958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:51:49.370984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 585/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:49.376134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff32c0b-60ef-4c5d-b2e7-0d4b8dd055d2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:49.464073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:49.472954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff32c0b-60ef-4c5d-b2e7-0d4b8dd055d2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:49.506367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:49.507018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:49.507336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:49.507616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:49.508366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:49.508676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:49.510048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:49.518455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 604/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:49.524333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:49.592553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:49.622400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76c90-1b93-49f5-a0d3-1d30c09d5a2e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:49.656276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:49.657132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:49.657418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:49.657697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] PATH_INFO: `/v3/users/39a726ead61541a2af734994c25444f2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:49.659335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=39a726ead61541a2af734994c25444f2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:49.660760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:49.662370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 586/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:49 2022] GET /identity/v3/users/39a726ead61541a2af734994c25444f2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:49.668210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:49.768783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac821af8-e548-455e-86d3-3d8d4042b81a None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:49.801399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634477b-77a9-4a02-9b23-c0a645df43c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:49.802237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634477b-77a9-4a02-9b23-c0a645df43c7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:49.802518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634477b-77a9-4a02-9b23-c0a645df43c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:49.802791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634477b-77a9-4a02-9b23-c0a645df43c7 None admin] PATH_INFO: `/v3/projects/07756f54a9f4427ca491d06dfbccbc0d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:49.803973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634477b-77a9-4a02-9b23-c0a645df43c7 None admin] RBAC: Authorizing `identity:get_project(project_id=07756f54a9f4427ca491d06dfbccbc0d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:49.805413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634477b-77a9-4a02-9b23-c0a645df43c7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:49.807078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 605/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:49 2022] GET /identity/v3/projects/07756f54a9f4427ca491d06dfbccbc0d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:49.813133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:49.910396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee213c-9d86-4bf6-a524-487f199f8dd9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:49.943136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b140e-1f9f-481e-b542-fec91f093a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:49.944041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b140e-1f9f-481e-b542-fec91f093a62 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:49.944325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b140e-1f9f-481e-b542-fec91f093a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:49.944604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b140e-1f9f-481e-b542-fec91f093a62 None admin] PATH_INFO: `/v3/projects/07756f54a9f4427ca491d06dfbccbc0d/users/39a726ead61541a2af734994c25444f2/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:49.946507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b140e-1f9f-481e-b542-fec91f093a62 None admin] RBAC: Authorizing `identity:create_grant(project_id=07756f54a9f4427ca491d06dfbccbc0d, user_id=39a726ead61541a2af734994c25444f2, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:49.948031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b140e-1f9f-481e-b542-fec91f093a62 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:50.026590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:50.031857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 587/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:51:49 2022] PUT /identity/v3/projects/07756f54a9f4427ca491d06dfbccbc0d/users/39a726ead61541a2af734994c25444f2/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:51:52.019962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88bece-c513-4958-9944-59f3d5541fab None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:52.020481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88bece-c513-4958-9944-59f3d5541fab None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:52.020590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88bece-c513-4958-9944-59f3d5541fab None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:52.021883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 606/1193] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:51:52.026903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01611-c89c-4283-9f8c-dab394212e6e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:52.027356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01611-c89c-4283-9f8c-dab394212e6e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:52.027496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01611-c89c-4283-9f8c-dab394212e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:52.043777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8ae01611-c89c-4283-9f8c-dab394212e6e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:51:52.044958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8ae01611-c89c-4283-9f8c-dab394212e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:51:52.110397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 588/1194] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:51:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:51:52.134343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88bece-c513-4958-9944-59f3d5541fab None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:52.158961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:52.233619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88bece-c513-4958-9944-59f3d5541fab None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:52.266349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:52.267118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:52.267461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:52.267738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:52.277446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:51:52.285475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:51:52.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:51:52.287349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 607/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:51:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:51:52.292513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae01611-c89c-4283-9f8c-dab394212e6e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:52.388450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae01611-c89c-4283-9f8c-dab394212e6e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:52.414463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:52.421933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:52.425483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:52.425483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:52.425483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:52.425483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:52.425483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:52.426159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:52.433990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 589/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:51:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:52.440013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:52.519856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:52.537608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4ff65-cbcd-4bd7-ac92-b536b9b454e4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:52.570385 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:52.571260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:52.571546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:52.571824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] PATH_INFO: `/v3/users/39a726ead61541a2af734994c25444f2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:52.572989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] RBAC: Authorizing `identity:get_user(user_id=39a726ead61541a2af734994c25444f2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:52.574405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:52.576020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 608/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 10:51:52 2022] GET /identity/v3/users/39a726ead61541a2af734994c25444f2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:52.582307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:52.679564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2209434-c6fb-40fb-b292-45ad0cde60b7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:52.711914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:52.712552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:52.712840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:52.713120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:52.768063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:52.779976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:52.779976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:52.875112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 590/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:51:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:51:52.881061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:52.980534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ea87a-e4a8-4ae8-8534-a7ac0c7d5832 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:53.013195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf2930-7d68-45bb-a5ca-79d567c5ba31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:53.014051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf2930-7d68-45bb-a5ca-79d567c5ba31 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:53.014334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf2930-7d68-45bb-a5ca-79d567c5ba31 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:53.014608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf2930-7d68-45bb-a5ca-79d567c5ba31 None admin] PATH_INFO: `/v3/projects/07756f54a9f4427ca491d06dfbccbc0d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:53.015784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf2930-7d68-45bb-a5ca-79d567c5ba31 None admin] RBAC: Authorizing `identity:get_project(project_id=07756f54a9f4427ca491d06dfbccbc0d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:53.017205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf2930-7d68-45bb-a5ca-79d567c5ba31 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:53.019464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 609/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 10:51:52 2022] GET /identity/v3/projects/07756f54a9f4427ca491d06dfbccbc0d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:51:53.025495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:51:53.111633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:51:53.120626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf56b2-0b21-4f27-a53d-19eecbcb9e9a None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:51:53.154349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d4c3f-1f75-4340-a78a-c87c00cd615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:51:53.154998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d4c3f-1f75-4340-a78a-c87c00cd615c None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:51:53.155319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d4c3f-1f75-4340-a78a-c87c00cd615c None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:51:53.155604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d4c3f-1f75-4340-a78a-c87c00cd615c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:51:53.156437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d4c3f-1f75-4340-a78a-c87c00cd615c None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=07756f54a9f4427ca491d06dfbccbc0d, user.id=39a726ead61541a2af734994c25444f2) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:51:53.156756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d4c3f-1f75-4340-a78a-c87c00cd615c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:51:53.158179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d4c3f-1f75-4340-a78a-c87c00cd615c None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:51:53.163920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 591/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:51:53 2022] GET /identity/v3/role_assignments?user.id=39a726ead61541a2af734994c25444f2&scope.project.id=07756f54a9f4427ca491d06dfbccbc0d&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:52:03.009168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:52:03.161246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=23 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:13.976653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c8fc-b721-4256-a7ca-9454cc465c66 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:13.977205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c8fc-b721-4256-a7ca-9454cc465c66 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:13.977205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c8fc-b721-4256-a7ca-9454cc465c66 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:13.978586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 610/1201] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:13.983754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683539a-c89d-46c5-a8b1-62015ea5c28b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:13.984196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683539a-c89d-46c5-a8b1-62015ea5c28b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:13.984322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683539a-c89d-46c5-a8b1-62015ea5c28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:14.000890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4683539a-c89d-46c5-a8b1-62015ea5c28b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:14.004043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4683539a-c89d-46c5-a8b1-62015ea5c28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:14.067867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 592/1202] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:14.092759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c8fc-b721-4256-a7ca-9454cc465c66 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:14.194163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c8fc-b721-4256-a7ca-9454cc465c66 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:14.228411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:14.229201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:14.229497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:14.229780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:14.240360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:14.241807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:14.259571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:14.260805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:14.261794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:14.261794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:14.261794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:14.261794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:14.261794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 611/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:14.266960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683539a-c89d-46c5-a8b1-62015ea5c28b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:14.367195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683539a-c89d-46c5-a8b1-62015ea5c28b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:14.394609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:14.400525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02518f20-a615-450b-afd2-855484f18ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:14.401185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518f20-a615-450b-afd2-855484f18ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:14.401471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518f20-a615-450b-afd2-855484f18ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:14.401750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02518f20-a615-450b-afd2-855484f18ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:14.402469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02518f20-a615-450b-afd2-855484f18ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:14.402779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02518f20-a615-450b-afd2-855484f18ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:14.404170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02518f20-a615-450b-afd2-855484f18ff0 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:14.415695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 593/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:14.421426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:14.474963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:14.520891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0457f-83af-4cb6-b93c-8f68bdefc227 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:14.554252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b6a42-c6e4-4e34-9da1-c943bb47994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:14.554916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6a42-c6e4-4e34-9da1-c943bb47994d None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:14.555227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6a42-c6e4-4e34-9da1-c943bb47994d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:14.555512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6a42-c6e4-4e34-9da1-c943bb47994d None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:14.556213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b6a42-c6e4-4e34-9da1-c943bb47994d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:14.557651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b6a42-c6e4-4e34-9da1-c943bb47994d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:14.707889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 612/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 10:53:14 2022] POST /identity/v3/users => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:53:16.775043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dcb1f-1466-4b15-85a0-6bf175731c58 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:16.775653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dcb1f-1466-4b15-85a0-6bf175731c58 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:16.775740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086dcb1f-1466-4b15-85a0-6bf175731c58 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:16.777080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 594/1206] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:16.782514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5147c-ddae-4981-ba7f-d6b95bc05552 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:16.782973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5147c-ddae-4981-ba7f-d6b95bc05552 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:16.783074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5147c-ddae-4981-ba7f-d6b95bc05552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:16.799799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-36f5147c-ddae-4981-ba7f-d6b95bc05552 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:16.800974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-36f5147c-ddae-4981-ba7f-d6b95bc05552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:16.866940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 613/1207] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:16.891049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086dcb1f-1466-4b15-85a0-6bf175731c58 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:16.991492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086dcb1f-1466-4b15-85a0-6bf175731c58 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:17.024292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:17.025130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:17.025374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:17.025654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:17.035441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:53:17.043106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:17.044301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:17.045211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 595/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:53:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:17.050424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5147c-ddae-4981-ba7f-d6b95bc05552 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:17.090544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:17.149812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5147c-ddae-4981-ba7f-d6b95bc05552 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:17.182973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:17.183661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:17.183943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:17.184218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:17.185025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:17.185261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:17.186673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:17.194421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 614/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:53:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:17.200162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:17.250390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:17.299659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a600d1-1117-4b85-bbcb-71f67e971a8b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:17.332291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:17.333257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:17.333636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:17.333921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:17.344401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:17.345841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:17.363207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:17.364759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:17.366134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:17.366134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:17.366134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:17.366134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:17.366134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 596/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:17.371109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:17.468657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:17.471586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c664bb-a28e-4aeb-bd22-d3aade256c69 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:17.505314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:17.505978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:17.506303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:17.506571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:17.507321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:17.507626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:17.509195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:17.520126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 615/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:17.525798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:17.626929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45431e-ca34-41a9-9a22-14b55070c75e None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:17.628478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:17.660399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:17.661401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:17.661691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:17.661970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:17.668771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:17.670322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:53:17.685370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:17.686552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:17.687869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 597/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:17.693715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:17.792799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f79706-b57c-41f6-8652-0297f26b78ad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:17.827567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:17.828243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:17.828524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:17.828802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:17.829628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:17.829993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:17.831432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:17.849633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 616/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 10:53:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:17.855605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:17.955544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e6430-bbd2-45a2-87fa-a5bd18c4475b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:17.990112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:17.993258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:17.993258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:17.993258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:18.020296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:18.057003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:18.058459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:18.155190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 598/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:53:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:53:18.161214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:18.179860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:18.260076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303624b1-de60-4a30-951f-bb3136625831 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:18.292471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:18.293260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:18.293826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:18.294109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:18.304405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:18.305836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:18.323120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:18.326584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:18.327748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:18.327748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:18.327748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:18.327748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:18.327748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 617/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:18.330225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:18.390988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:18.428593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e83828-9bdd-415f-abae-7d85f327aa36 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:18.462629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:18.462629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:18.462629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:18.462629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:18.463314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:18.463622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:18.464963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:18.476654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 599/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:18.482425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:18.559664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:18.582270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564fc9c-acd5-4f5b-93c9-f1b14de975ad None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:18.616638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:18.617421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:18.617705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:18.617980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:18.624623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:18.626053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:53:18.639751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:18.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:18.642184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 618/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:53:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:18.646803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:18.748758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1067ac-a250-4a57-901b-a8a4ac659712 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:18.776371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:18.788242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228de7d-2f85-4b35-a849-aaab35ca6992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:18.788901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228de7d-2f85-4b35-a849-aaab35ca6992 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:18.789186 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228de7d-2f85-4b35-a849-aaab35ca6992 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:18.789462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228de7d-2f85-4b35-a849-aaab35ca6992 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:18.790242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228de7d-2f85-4b35-a849-aaab35ca6992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:18.790551 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228de7d-2f85-4b35-a849-aaab35ca6992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:18.792059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228de7d-2f85-4b35-a849-aaab35ca6992 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:18.804001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 600/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:53:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:53:18.810008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:18.908383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4dae0-c914-4595-abd0-d51f0f82e02d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:18.939471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:18.945085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623397f4-c986-467c-ac80-3759df216503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:18.945739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623397f4-c986-467c-ac80-3759df216503 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:18.946024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623397f4-c986-467c-ac80-3759df216503 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:18.946299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623397f4-c986-467c-ac80-3759df216503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:18.947126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623397f4-c986-467c-ac80-3759df216503 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=80003dcc49ef47c8a08887462d102b5d) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:18.947494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623397f4-c986-467c-ac80-3759df216503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:18.948918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623397f4-c986-467c-ac80-3759df216503 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:18.954118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 619/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:53:18 2022] GET /identity/v3/role_assignments?user.id=80003dcc49ef47c8a08887462d102b5d&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:20.960659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9357-4693-4ea7-abfb-97175e4393bc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:20.961189 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9357-4693-4ea7-abfb-97175e4393bc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:20.961295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9357-4693-4ea7-abfb-97175e4393bc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:20.962875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 601/1220] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:20.968123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2ba8-313e-4e12-8a8d-42023ada7940 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:20.968534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2ba8-313e-4e12-8a8d-42023ada7940 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:20.968686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2ba8-313e-4e12-8a8d-42023ada7940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:20.985897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4f3f2ba8-313e-4e12-8a8d-42023ada7940 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:20.987071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4f3f2ba8-313e-4e12-8a8d-42023ada7940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:21.053084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 620/1221] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:21.076130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9357-4693-4ea7-abfb-97175e4393bc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:21.182417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9357-4693-4ea7-abfb-97175e4393bc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:21.216593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:21.217374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:21.217658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:21.217936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:21.228648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:53:21.236373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:21.237468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:21.238527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 602/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:53:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:21.244195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2ba8-313e-4e12-8a8d-42023ada7940 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:21.343166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2ba8-313e-4e12-8a8d-42023ada7940 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:21.367392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:21.377532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:21.378194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:21.378479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:21.378923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:21.379619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] RBAC: Adding query filter params (name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:21.379928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:21.381389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:21.389203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 621/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:53:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:21.394915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:21.438319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:21.494106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c761ae5-58d7-4a44-81a0-9bbef0fc965d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:21.528057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:21.528847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:21.529165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:21.529441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:21.540967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:21.542403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:21.559774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:21.560815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:21.561741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:21.561741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:21.561741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:21.561741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:21.561741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 603/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:21.566735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:21.668780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae0d5a-5e3f-4438-9e68-c26c4c71688b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:21.703553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:21.704126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:21.704411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:21.704688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:21.705410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:21.705720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:21.707080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:21.719956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 622/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:21.726450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:21.819338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:21.825824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8465-708d-4100-a22b-a60a8e4df856 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:21.858710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:21.859537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:21.859823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:21.860098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:21.866963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:21.868441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:53:21.883495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:21.884616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:21.889696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 604/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:21.890436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:21.986507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:21.993547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75f10-9855-4458-bcb4-c7adfa3723e0 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:22.028340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:22.029014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:22.029328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:22.029609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:22.030450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:22.030765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:22.032231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:22.050275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 623/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 10:53:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:22.056914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:22.162223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccec0f-4e35-429a-a76d-4d1e0e121d1b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:22.196906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:22.197691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:22.197976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:22.198253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:22.208286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:22.209836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:22.227389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:22.228577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:22.229623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:22.229623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:22.229623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:22.229623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:22.229623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 605/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:22.234308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:22.332253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94557ed1-0b4b-47b9-bc69-8e295a3cf2e7 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:22.334780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:22.366830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:22.367579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:22.367868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:22.368147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:22.368876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:22.369188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:22.370558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:22.381801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 624/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:22.387395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:22.396194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:22.486333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03024-e241-4524-903a-8061fd9c4b36 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:22.519038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:22.519849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:22.520603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:22.520841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:22.527286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:22.528727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:53:22.543482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:22.544534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:22.545634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 606/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:22.550629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:22.650011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bcbd7-fb83-4ed4-84a3-e63c6fecc700 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:22.692865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b055a1-a7d1-4d6a-b2f1-2116c99e009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:22.693541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b055a1-a7d1-4d6a-b2f1-2116c99e009f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:22.693832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b055a1-a7d1-4d6a-b2f1-2116c99e009f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:22.694113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b055a1-a7d1-4d6a-b2f1-2116c99e009f None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:22.694910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b055a1-a7d1-4d6a-b2f1-2116c99e009f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:22.695298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b055a1-a7d1-4d6a-b2f1-2116c99e009f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:22.696760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b055a1-a7d1-4d6a-b2f1-2116c99e009f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:22.709945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 625/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:22.716159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:22.781781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:22.815274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8a512-5b3f-43ae-8a1d-ea9bb21ae8f4 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:22.847870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e79a48-f70d-4319-babc-55699e65bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:22.849400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e79a48-f70d-4319-babc-55699e65bee7 None admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:22.849689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e79a48-f70d-4319-babc-55699e65bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:22.849970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e79a48-f70d-4319-babc-55699e65bee7 None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/80003dcc49ef47c8a08887462d102b5d/roles/4fc9795c5001472c86ae32739f03b3f6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:22.863902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e79a48-f70d-4319-babc-55699e65bee7 None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=80003dcc49ef47c8a08887462d102b5d, role_id=4fc9795c5001472c86ae32739f03b3f6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:22.865348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e79a48-f70d-4319-babc-55699e65bee7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:22.993771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 607/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:53:22 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/80003dcc49ef47c8a08887462d102b5d/roles/4fc9795c5001472c86ae32739f03b3f6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:53:25.018453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6e4d-5d13-4b2e-a1c7-a05e7c822aa1 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:25.018996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6e4d-5d13-4b2e-a1c7-a05e7c822aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:25.018996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6e4d-5d13-4b2e-a1c7-a05e7c822aa1 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:25.021001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 626/1233] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:25.026262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d3eb3-fb6e-4c79-a02f-3ef75d32a93a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:25.026730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d3eb3-fb6e-4c79-a02f-3ef75d32a93a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:25.026876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d3eb3-fb6e-4c79-a02f-3ef75d32a93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:25.043942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-400d3eb3-fb6e-4c79-a02f-3ef75d32a93a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:25.045135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-400d3eb3-fb6e-4c79-a02f-3ef75d32a93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:25.111664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 608/1234] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:25.136017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6e4d-5d13-4b2e-a1c7-a05e7c822aa1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:25.162216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:25.236376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6e4d-5d13-4b2e-a1c7-a05e7c822aa1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:25.271062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:25.271880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:25.272168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:25.272449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:25.282337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:53:25.290195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:25.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:25.293067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 627/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 10:53:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:25.297552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d3eb3-fb6e-4c79-a02f-3ef75d32a93a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:25.344631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:25.408392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d3eb3-fb6e-4c79-a02f-3ef75d32a93a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:25.442022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:25.442687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:25.442975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:25.443281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:25.444086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] RBAC: Adding query filter params (name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:25.444391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:25.445745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:25.453380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 609/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 10:53:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:25.459168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:25.547813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:25.564739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602e8a7-a253-48ab-bbdd-5e5e701dd98d None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:25.600574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:25.601375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:25.601659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:25.601938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:25.613081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:25.615292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:25.633315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:25.634447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:25.635490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:25.635490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:25.635490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:25.635490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:25.635490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 628/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 10:53:25.640861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:25.755553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbd29b-20db-43b9-9793-f09a7b1af439 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:25.767403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:25.795791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:25.796463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:25.796755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:25.797041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:25.797878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:25.798147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:25.799539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:25.813609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 610/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:25.819444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:25.936137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65427e-d27b-4872-a210-dd7170c98740 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:25.970165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:25.970967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:25.971284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:25.971566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:25.974528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:25.979860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:25.981347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:53:25.996863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:25.998708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:25.999735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 629/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:26.003990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:26.114475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa200df-b260-4283-9b73-4aea1a2309fb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:26.151142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:26.151889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:26.152206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:26.152540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:26.153321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:26.153632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:26.155291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:26.173046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 611/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 10:53:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:26.179628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:26.288644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216f804-f625-4d2a-911e-1f870cea5f85 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:26.321353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:26.321996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:26.322283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:26.322560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:26.383393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:26.389606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:26.391045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:26.488249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 630/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:53:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:53:26.494766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:26.523600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:26.593534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454985-9bc4-4ba3-82dd-7f3ae070c2c8 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:26.626535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:26.627391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:26.627643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:26.627921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:26.638440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:26.639838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:26.657637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:26.658741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:26.664708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:26.664708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:26.664708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:26.664708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:26.664708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 612/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:26.666613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:26.760410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:26.766006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9bcbb-b7dc-4df1-ace6-c02205b577c4 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:26.798192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:26.798956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:26.799271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:26.799549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:26.800268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:26.800576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:26.802525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:26.813478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 631/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:26.820313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:26.900974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:26.922388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91db5e9-cc6a-452d-a99c-dadff5325f41 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:26.956030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:26.956821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:26.957108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:26.957387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:26.964282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:26.965715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:53:26.980400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:26.984664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:26.985594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 613/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:26.987440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:27.086390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf46e7-6c99-4959-b0b5-b4299b233f19 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:27.120480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d834-ad21-4daa-9011-2dd465abb6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:27.121146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d834-ad21-4daa-9011-2dd465abb6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:27.121431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d834-ad21-4daa-9011-2dd465abb6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:27.121706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d834-ad21-4daa-9011-2dd465abb6a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:27.122483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d834-ad21-4daa-9011-2dd465abb6a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:27.122790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d834-ad21-4daa-9011-2dd465abb6a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:27.124495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d834-ad21-4daa-9011-2dd465abb6a2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:27.136451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 632/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:27.142764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:27.243308 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba4ac-1499-4594-9607-dad4738db11b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:27.277041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bce740-1d77-4744-b3a5-166f260526e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:27.277682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce740-1d77-4744-b3a5-166f260526e4 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:27.277968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce740-1d77-4744-b3a5-166f260526e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:27.278246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce740-1d77-4744-b3a5-166f260526e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:27.279070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bce740-1d77-4744-b3a5-166f260526e4 None admin] RBAC: Adding query filter params (role.id=4fc9795c5001472c86ae32739f03b3f6, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=80003dcc49ef47c8a08887462d102b5d) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:27.279416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bce740-1d77-4744-b3a5-166f260526e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:27.280836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bce740-1d77-4744-b3a5-166f260526e4 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:27.287233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 614/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:53:27 2022] GET /identity/v3/role_assignments?user.id=80003dcc49ef47c8a08887462d102b5d&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=4fc9795c5001472c86ae32739f03b3f6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:29.385177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184850a-9610-4b49-9404-d5fb30381625 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:29.385726 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184850a-9610-4b49-9404-d5fb30381625 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:29.385726 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184850a-9610-4b49-9404-d5fb30381625 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:29.387224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 633/1247] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:29.392465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17718-de83-4031-94b9-46d80ca901ac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:29.392872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17718-de83-4031-94b9-46d80ca901ac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:29.393070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17718-de83-4031-94b9-46d80ca901ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:29.398879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:29.411024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-63f17718-de83-4031-94b9-46d80ca901ac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:29.412259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-63f17718-de83-4031-94b9-46d80ca901ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:29.478890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 615/1248] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:29.503631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184850a-9610-4b49-9404-d5fb30381625 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:29.522646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:29.603813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184850a-9610-4b49-9404-d5fb30381625 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:29.637636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:29.638360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:29.638646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:29.638923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:29.649144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:53:29.656690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:29.657872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:29.664094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 634/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:29.664845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17718-de83-4031-94b9-46d80ca901ac None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:29.762622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17718-de83-4031-94b9-46d80ca901ac None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:29.790753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:29.795473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:29.796137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:29.796424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:29.796704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:29.797450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:29.797761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:29.799129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:29.808286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 616/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:29.813962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:29.891062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:29.913995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f170b7-daf0-413a-979c-bbbbf9466029 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:29.947421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:29.948213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:29.948499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:29.948776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:29.959162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:29.964710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:29.978221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:29.979346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:29.980299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:29.980299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:29.980299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:29.980299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:29.980299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 635/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:29.985223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:30.082564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0852f-8e2b-491c-9ff8-c04670bd0fab None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:30.116336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:30.117027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:30.117320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:30.117604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:30.118397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:30.118712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:30.120106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:30.132467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 617/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:30.138151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:30.240110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada6d5f-7978-43c2-84ac-3e06422b5141 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:30.264846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:30.273016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:30.273805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:30.274088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:30.274362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:30.281484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:30.282910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:53:30.297475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:30.298596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:30.299976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 636/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:30.304899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:30.360475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:30.403709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef3b0c-cc69-4fc9-8e0c-9ebca0602355 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:30.436457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:30.437118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:30.437406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:30.437711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:30.438581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:30.438896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:30.440353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:30.458251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 618/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 10:53:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:30.464507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:30.566673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3acfc-9d3c-428f-9ce0-eaf63f25afae None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:30.603476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:30.604181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:30.604527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:30.604890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:30.666156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:30.678507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:30.680005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:30.782724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 637/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:53:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:53:30.788573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:30.886717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a804a67-72c0-4979-a453-feda36855f7d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:30.899952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:30.921152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:30.921915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:30.922200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:30.922477 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:30.932755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:30.934481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:30.952452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:30.953539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:30.954624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:30.954624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:30.954624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:30.954624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:30.954624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 619/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:30.960005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:31.045616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:31.057539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe515e3-fb12-4381-ab0d-b81e36740ef8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:31.090905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:31.091598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:31.091891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:31.092177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:31.092905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:31.093219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:31.094586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:31.105527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 638/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:31.111172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:31.210779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564e0e2-20da-4c12-b6e2-aa4057747e08 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:31.244269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:31.245060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:31.245345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:31.245624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:31.252085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:31.253514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:53:31.267489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:31.279653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:31.280828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 620/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:53:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:31.281450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:31.375198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013f70b-133c-45d3-89ed-46c9529e3b2f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:31.408721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ce7cc-895f-4383-8c92-94aad714ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:31.409380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ce7cc-895f-4383-8c92-94aad714ad6c None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:31.409673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ce7cc-895f-4383-8c92-94aad714ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:31.409955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ce7cc-895f-4383-8c92-94aad714ad6c None admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:31.411326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ce7cc-895f-4383-8c92-94aad714ad6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:31.411647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ce7cc-895f-4383-8c92-94aad714ad6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:31.413081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ce7cc-895f-4383-8c92-94aad714ad6c None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:31.424757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 639/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:31.431410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:31.456446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:31.529447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe0ef4-774f-4233-8efa-6c4c2fe85ea9 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:31.562883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1ff41-d7a5-4374-9a3a-9b8a4c38ad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:31.563587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ff41-d7a5-4374-9a3a-9b8a4c38ad92 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:31.563869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ff41-d7a5-4374-9a3a-9b8a4c38ad92 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:31.564151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1ff41-d7a5-4374-9a3a-9b8a4c38ad92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:31.564975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1ff41-d7a5-4374-9a3a-9b8a4c38ad92 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=80003dcc49ef47c8a08887462d102b5d) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:31.565293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1ff41-d7a5-4374-9a3a-9b8a4c38ad92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:31.566760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1ff41-d7a5-4374-9a3a-9b8a4c38ad92 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:31.572140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 621/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:53:31 2022] GET /identity/v3/role_assignments?user.id=80003dcc49ef47c8a08887462d102b5d&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:33.571321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3054da-6c74-4a9d-90fd-957cb37f9163 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:33.571850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3054da-6c74-4a9d-90fd-957cb37f9163 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:33.571850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3054da-6c74-4a9d-90fd-957cb37f9163 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:33.573238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 640/1261] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:33.578353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5ab6-2f75-432b-ad32-de7e8cd4cf6b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:33.578811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5ab6-2f75-432b-ad32-de7e8cd4cf6b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:33.578980 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c5ab6-2f75-432b-ad32-de7e8cd4cf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:33.596471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0c3c5ab6-2f75-432b-ad32-de7e8cd4cf6b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:33.597680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0c3c5ab6-2f75-432b-ad32-de7e8cd4cf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:33.666769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 622/1262] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:33.690295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3054da-6c74-4a9d-90fd-957cb37f9163 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:33.701612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:33.791568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3054da-6c74-4a9d-90fd-957cb37f9163 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:33.823580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:33.824312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:33.824599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:33.824877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:33.835153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:53:33.843020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:33.844256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:33.845265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 641/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:53:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:33.851054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c5ab6-2f75-432b-ad32-de7e8cd4cf6b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:33.869181 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:33.947408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c5ab6-2f75-432b-ad32-de7e8cd4cf6b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:33.981537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:33.982200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:33.982485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:33.982765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:33.983532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:33.983844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:33.985238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:33.993109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 623/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:53:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:33.998837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:34.063873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:34.096305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9c22e-8eb2-4d65-931e-7a000356feb5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:34.128431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:34.133382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:34.133382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:34.133382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:34.142837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:34.144311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:34.161652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:34.162764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:34.168321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:34.168321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:34.168321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:34.168321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:34.168321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 642/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:34.169137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:34.252345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:34.272362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566c7f9-2a6c-40b9-a6c0-ba03c81c84d7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:34.304919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:34.305572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:34.305858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:34.306136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:34.306853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:34.307170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:34.308565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:34.319862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 624/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:34.325573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:34.425096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4babfe-4e6d-4261-873d-9f0a481d8cd8 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:34.444652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:34.457989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:34.458787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:34.459071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:34.459374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:34.467904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:34.471697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:53:34.483962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:34.484956 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:34.485966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 643/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:34.491572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:34.590808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfc8e7-5f77-4d2b-98ac-3ffeec4e8140 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:34.623452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:34.624205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:34.624528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:34.624812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:34.625661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:34.625984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:34.627490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:34.638377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:34.646448 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 625/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 10:53:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 10:53:34.652167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:34.753198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71623952-c687-46b3-a5e8-95dbacafe29f None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:34.791285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:34.792077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:34.792361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:34.792635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:34.803479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:34.804905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:34.821922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:34.823070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:34.826082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:34.826082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:34.826082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:34.826082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:34.826082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 644/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:34.828785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:34.928054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4c16a-938b-414b-814e-0ca3b563d3d2 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:34.961327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:34.962814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:34.962814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:34.962814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:34.963539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:34.963850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:34.965212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:34.976972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 626/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:34.982767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:35.014457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:35.080133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78cb80-991a-4d00-b168-7db08b1beedf None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:35.112745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:35.114159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:35.114451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:35.114722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:35.121070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:35.122514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:53:35.137708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:35.139237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:35.140715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 645/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:35.145024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:35.206898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:35.249073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef04b0-8c43-4ec4-b576-b227403cb11b None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:35.283433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23e8d4-391b-45b1-b25e-fd92c7150c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:35.283433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23e8d4-391b-45b1-b25e-fd92c7150c8b None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:35.283433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23e8d4-391b-45b1-b25e-fd92c7150c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:35.283433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23e8d4-391b-45b1-b25e-fd92c7150c8b None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:35.283861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23e8d4-391b-45b1-b25e-fd92c7150c8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:35.283963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23e8d4-391b-45b1-b25e-fd92c7150c8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:35.285961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23e8d4-391b-45b1-b25e-fd92c7150c8b None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:35.297749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 627/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:35.303735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:35.393253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:35.402177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4aace6-c40f-4281-ae33-af01bc111b18 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:35.436610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b03c01-1c14-4f97-b19d-4e12af40eefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:35.437525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b03c01-1c14-4f97-b19d-4e12af40eefe None admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:35.437826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b03c01-1c14-4f97-b19d-4e12af40eefe None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:35.438102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b03c01-1c14-4f97-b19d-4e12af40eefe None admin] PATH_INFO: `/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/80003dcc49ef47c8a08887462d102b5d/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:35.440085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b03c01-1c14-4f97-b19d-4e12af40eefe None admin] RBAC: Authorizing `identity:create_grant(project_id=87b0060a49c54aa79beba6f2a83ea381, user_id=80003dcc49ef47c8a08887462d102b5d, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:35.441638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b03c01-1c14-4f97-b19d-4e12af40eefe None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:35.588665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 646/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 10:53:35 2022] PUT /identity/v3/projects/87b0060a49c54aa79beba6f2a83ea381/users/80003dcc49ef47c8a08887462d102b5d/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 10:53:37.594394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8d30-bd48-4cdd-8651-f8e60a155b1c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:37.594958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8d30-bd48-4cdd-8651-f8e60a155b1c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:37.594958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8d30-bd48-4cdd-8651-f8e60a155b1c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:37.596437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 628/1274] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:37.601513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce85d6-aafe-49f8-ad6c-5fca640c8f20 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:37.601951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce85d6-aafe-49f8-ad6c-5fca640c8f20 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:37.602063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce85d6-aafe-49f8-ad6c-5fca640c8f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:37.621037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ccce85d6-aafe-49f8-ad6c-5fca640c8f20 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:37.622228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ccce85d6-aafe-49f8-ad6c-5fca640c8f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:37.687337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 647/1275] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:37.713029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8d30-bd48-4cdd-8651-f8e60a155b1c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:37.812366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8d30-bd48-4cdd-8651-f8e60a155b1c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:37.830889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:37.847380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:37.848112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:37.848395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:37.848674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:37.858355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 10:53:37.866490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:37.867671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:37.868832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 629/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 10:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:37.873753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce85d6-aafe-49f8-ad6c-5fca640c8f20 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:37.954602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:37.972084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce85d6-aafe-49f8-ad6c-5fca640c8f20 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:38.005198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:38.005861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:38.006147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:38.006425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:38.007209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:38.007525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:38.008892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:38.016605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 648/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 10:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:38.022236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:38.121164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfeb368-157e-478f-ba02-4cced70a3eff None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:38.155157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:38.155939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:38.156225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:38.156509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:38.167539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:38.168972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:38.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:38.187165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:38.188217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:38.188217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:38.188217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:38.188217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:38.188217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 630/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:38.193599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:38.292527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79bc92-41ad-408e-b73f-54b635fe776e None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:38.325777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:38.326436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:38.326732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:38.327013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:38.327766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:38.328078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:38.329455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:38.341355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 649/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:38.346989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:38.379789 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:38.447561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11529268-2978-433e-b61b-896f375d15d7 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:38.480605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:38.481399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:38.481688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:38.481970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:38.489276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:38.490783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 10:53:38.505997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:38.507040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:38.508194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 631/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:38.513005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:38.516740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:38.608347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ba36-c650-4359-bb75-a6ff1defc28b None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:38.641526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:38.642131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:38.642410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:38.642684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:38.643620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:38.644135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:38.646911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:38.663966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 650/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 10:53:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:38.669825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:38.760104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:38.769317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece22083-f942-4690-a1b0-2c4496b058eb None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:38.802709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:38.803375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:38.803662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:38.803941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:38.870670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:38.872138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:38.960596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:38.970999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 632/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 10:53:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 10:53:38.977362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:39.038703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:39.075430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b5fa-b1da-4bcb-9c7e-ebad51ce0960 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:39.109794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:39.111359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:39.111359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:39.111359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:39.120990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:39.122458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 10:53:39.139687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:39.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:39.141586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 10:53:39.141586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 10:53:39.141586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 10:53:39.141586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:39.141586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 651/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 10:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:39.147460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:39.245313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e60781-d9e1-4781-a693-6a39b3ad2282 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:39.278533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:39.279221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:39.279516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:39.279802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:39.280523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:39.280839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:39.282212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:39.293771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 633/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 10:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:39.299420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:39.399021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1e5f-9745-497e-9d53-4cc3afdaee84 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:39.412553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:39.431608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:39.432417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:39.432701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:39.432974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:39.439317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:39.440778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 10:53:39.456783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:39.457907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 10:53:39.459044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 652/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 10:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:39.464588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:39.513837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:39.561044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbc0df-3a4f-4fd6-9af9-8e314da1500d None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:39.593895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69e67-a086-4137-85c0-59805fb90a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:39.594558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69e67-a086-4137-85c0-59805fb90a20 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:39.594848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69e67-a086-4137-85c0-59805fb90a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:39.595133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69e67-a086-4137-85c0-59805fb90a20 None admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:39.595951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69e67-a086-4137-85c0-59805fb90a20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:39.596266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69e67-a086-4137-85c0-59805fb90a20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:39.597765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69e67-a086-4137-85c0-59805fb90a20 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:39.609472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 634/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 10:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:39.615505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:39.713213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856466c-0322-4792-8815-9f8d2f956828 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:39.746047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce855a5a-19de-4e08-bc3a-d375e91789c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:39.746695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce855a5a-19de-4e08-bc3a-d375e91789c2 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:39.746979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce855a5a-19de-4e08-bc3a-d375e91789c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:39.747283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce855a5a-19de-4e08-bc3a-d375e91789c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:39.748135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce855a5a-19de-4e08-bc3a-d375e91789c2 None admin] RBAC: Adding query filter params (role.id=dd9dd4b6e3c14eb4804574a721ad656b, scope.project.id=87b0060a49c54aa79beba6f2a83ea381, user.id=80003dcc49ef47c8a08887462d102b5d) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:39.748445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce855a5a-19de-4e08-bc3a-d375e91789c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:39.749890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce855a5a-19de-4e08-bc3a-d375e91789c2 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:39.756134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 653/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 10:53:39 2022] GET /identity/v3/role_assignments?user.id=80003dcc49ef47c8a08887462d102b5d&scope.project.id=87b0060a49c54aa79beba6f2a83ea381&role.id=dd9dd4b6e3c14eb4804574a721ad656b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:41.819420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435400c-2f18-4288-9f76-da4b42e0eb4b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:41.819954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435400c-2f18-4288-9f76-da4b42e0eb4b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:41.819954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435400c-2f18-4288-9f76-da4b42e0eb4b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:41.821554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 635/1288] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:41.826901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bf07b-f64d-486b-873d-95447b7cddbe None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:41.831692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bf07b-f64d-486b-873d-95447b7cddbe None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:41.831692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bf07b-f64d-486b-873d-95447b7cddbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:41.844850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ad4bf07b-f64d-486b-873d-95447b7cddbe None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:41.846014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ad4bf07b-f64d-486b-873d-95447b7cddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:41.874522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:41.912861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 654/1289] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 10:53:41.935144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435400c-2f18-4288-9f76-da4b42e0eb4b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:42.036526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435400c-2f18-4288-9f76-da4b42e0eb4b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:42.046797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:42.070316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:42.071056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:42.071371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:42.071652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:42.072345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:42.073686 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:42.085405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:42.092056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:42.093014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 636/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:53:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:42.093872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bf07b-f64d-486b-873d-95447b7cddbe None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:42.191007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bf07b-f64d-486b-873d-95447b7cddbe None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:42.225995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ede9f4-141b-4d18-8884-74a9b4dc578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:42.232330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ede9f4-141b-4d18-8884-74a9b4dc578f None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:42.232330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ede9f4-141b-4d18-8884-74a9b4dc578f None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:42.232330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ede9f4-141b-4d18-8884-74a9b4dc578f None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:42.232330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ede9f4-141b-4d18-8884-74a9b4dc578f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:42.232330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ede9f4-141b-4d18-8884-74a9b4dc578f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:42.232330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ede9f4-141b-4d18-8884-74a9b4dc578f None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:42.234405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 655/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 10:53:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:42.240851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:42.340082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b348b-4c91-4fbb-8f8b-71270c1f3826 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:42.372891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df1e90-defb-4bea-86f5-dad276bdc6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:42.373549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1e90-defb-4bea-86f5-dad276bdc6df None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:42.373832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1e90-defb-4bea-86f5-dad276bdc6df None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:42.374112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1e90-defb-4bea-86f5-dad276bdc6df None admin] PATH_INFO: `/v3/services` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:42.374810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1e90-defb-4bea-86f5-dad276bdc6df None admin] RBAC: Adding query filter params (type=placement) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:42.375116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1e90-defb-4bea-86f5-dad276bdc6df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:42.376489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1e90-defb-4bea-86f5-dad276bdc6df None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:42.381670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 637/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 10:53:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:44.396396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b52e8-acf4-4f54-978e-9c59b79d602d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:44.396924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b52e8-acf4-4f54-978e-9c59b79d602d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:44.397031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b52e8-acf4-4f54-978e-9c59b79d602d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:44.398352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 656/1293] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:44.403597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb077a-cbbd-4280-9fbf-9887cc151340 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:44.404023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb077a-cbbd-4280-9fbf-9887cc151340 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:44.404165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb077a-cbbd-4280-9fbf-9887cc151340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:44.421098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-fffb077a-cbbd-4280-9fbf-9887cc151340 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:44.422283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fffb077a-cbbd-4280-9fbf-9887cc151340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:44.453843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:44.492860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 638/1294] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:44.515118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b52e8-acf4-4f54-978e-9c59b79d602d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:44.547825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:44.617352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b52e8-acf4-4f54-978e-9c59b79d602d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:44.650695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9db83b-d4c6-4d01-8ec7-65a64f8ea9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:44.651324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9db83b-d4c6-4d01-8ec7-65a64f8ea9a1 None admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:44.651613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9db83b-d4c6-4d01-8ec7-65a64f8ea9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:44.651893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9db83b-d4c6-4d01-8ec7-65a64f8ea9a1 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:44.652644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9db83b-d4c6-4d01-8ec7-65a64f8ea9a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:44.654009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9db83b-d4c6-4d01-8ec7-65a64f8ea9a1 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:44.940114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 657/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 10:53:44 2022] POST /identity/v3/services => generated 246 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:53:46.954568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174b4a-ebb0-4654-9443-bf60d9435b24 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:46.955076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174b4a-ebb0-4654-9443-bf60d9435b24 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:46.955172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174b4a-ebb0-4654-9443-bf60d9435b24 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:46.956545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 639/1296] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:46.962113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984a61-c974-493e-a4a8-041878823863 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:46.962489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984a61-c974-493e-a4a8-041878823863 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:46.962677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8984a61-c974-493e-a4a8-041878823863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:46.979133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b8984a61-c974-493e-a4a8-041878823863 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:46.982234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b8984a61-c974-493e-a4a8-041878823863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:47.051981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 658/1297] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:47.075153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33174b4a-ebb0-4654-9443-bf60d9435b24 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:47.174419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33174b4a-ebb0-4654-9443-bf60d9435b24 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:47.208972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:47.209819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:47.210104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:47.210381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:47.211030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:47.212685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:47.224375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:47.225556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:47.226450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 640/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:53:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:47.231585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8984a61-c974-493e-a4a8-041878823863 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:47.250228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:47.329655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8984a61-c974-493e-a4a8-041878823863 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:47.366057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:47.381301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:47.381301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:47.381301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:47.381301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:47.381301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:47.381301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:47.381787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 659/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 10:53:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:47.381878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:47.411958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:47.479632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365034d-d608-4b6c-bdc4-2058a0ddb239 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:47.513172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e5d6f-d697-4f55-8fad-620f079e1169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:47.513826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e5d6f-d697-4f55-8fad-620f079e1169 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:47.514117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e5d6f-d697-4f55-8fad-620f079e1169 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:47.514404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e5d6f-d697-4f55-8fad-620f079e1169 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:47.515262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e5d6f-d697-4f55-8fad-620f079e1169 None admin] RBAC: Adding query filter params (interface=public, service_id=a6698140e25543e9a7adc4fb5f2cff6e, region_id=RegionOne) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:47.515546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e5d6f-d697-4f55-8fad-620f079e1169 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:47.517218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e5d6f-d697-4f55-8fad-620f079e1169 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:47.521984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 641/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 10:53:47 2022] GET /identity/v3/endpoints?service_id=a6698140e25543e9a7adc4fb5f2cff6e&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:47.527736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:47.626619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643bd2c-e9d7-400b-bf6a-3f6e683359b5 None admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:47.629516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:47.662077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596f1cd-9448-460a-a0d8-006ab7259b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:47.662748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596f1cd-9448-460a-a0d8-006ab7259b8d None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:47.663068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596f1cd-9448-460a-a0d8-006ab7259b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:47.663406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596f1cd-9448-460a-a0d8-006ab7259b8d None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:47.664107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596f1cd-9448-460a-a0d8-006ab7259b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:47.665504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596f1cd-9448-460a-a0d8-006ab7259b8d None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:47.672242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 660/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 10:53:47 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 10:53:49.667850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8349f5-8913-460e-9795-0cbe79294a87 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:49.668429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8349f5-8913-460e-9795-0cbe79294a87 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:49.668429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8349f5-8913-460e-9795-0cbe79294a87 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:49.669758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 642/1302] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 10:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 10:53:49.674959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58630-3395-4312-a0fa-d5f032338bfc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:49.675394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58630-3395-4312-a0fa-d5f032338bfc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:49.675941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba58630-3395-4312-a0fa-d5f032338bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:49.692916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3ba58630-3395-4312-a0fa-d5f032338bfc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 10:53:49.694093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3ba58630-3395-4312-a0fa-d5f032338bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 10:53:49.760882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 661/1303] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 10:53:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 10:53:49.784188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8349f5-8913-460e-9795-0cbe79294a87 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:49.877920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:49.884995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8349f5-8913-460e-9795-0cbe79294a87 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:49.921047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:49.921790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:49.922075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:49.922355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:49.923014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:49.924459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 10:53:49.936819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 10:53:49.938009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 10:53:49.939124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 643/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 10:53:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 10:53:49.944946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba58630-3395-4312-a0fa-d5f032338bfc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:50.018995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:53:50.040976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba58630-3395-4312-a0fa-d5f032338bfc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:50.074234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d959e29-9be9-4f9e-9b22-db774509f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:50.074893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d959e29-9be9-4f9e-9b22-db774509f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:50.075251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d959e29-9be9-4f9e-9b22-db774509f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:50.075536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d959e29-9be9-4f9e-9b22-db774509f5e8 None admin] PATH_INFO: `/v3/services` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:50.076244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d959e29-9be9-4f9e-9b22-db774509f5e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 10:53:50.076553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d959e29-9be9-4f9e-9b22-db774509f5e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:50.077923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d959e29-9be9-4f9e-9b22-db774509f5e8 None admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:50.083102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 662/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 10:53:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 10:53:50.089068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 10:53:50.188040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd859-bba4-461e-ab20-6ba316bd4b78 None admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 10:53:50.222655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39f98b-0a4f-4f89-bcfc-a66423b09ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 10:53:50.223438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f98b-0a4f-4f89-bcfc-a66423b09ceb None admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 10:53:50.223729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f98b-0a4f-4f89-bcfc-a66423b09ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 10:53:50.224014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f98b-0a4f-4f89-bcfc-a66423b09ceb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 10:53:50.224775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39f98b-0a4f-4f89-bcfc-a66423b09ceb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 10:53:50.226276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39f98b-0a4f-4f89-bcfc-a66423b09ceb None admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 10:53:50.322914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 644/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 10:53:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 10:54:00.139897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=30 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 10:54:00.317944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:50.187004 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f939ba-0b80-4500-958a-19cf2384bec0 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:50.187004 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f939ba-0b80-4500-958a-19cf2384bec0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:50.187004 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f939ba-0b80-4500-958a-19cf2384bec0 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:50.189232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a7864-b466-4828-9836-11d41622e5f1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:50.189893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a7864-b466-4828-9836-11d41622e5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:50.190046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a7864-b466-4828-9836-11d41622e5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:50.193232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 663/1307] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:01:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:50.279213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0e0a7864-b466-4828-9836-11d41622e5f1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:50.280723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0e0a7864-b466-4828-9836-11d41622e5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:50.412480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 645/1308] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:01:50.457997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9fba2-f365-4b4a-a50b-cd7530c88ef8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:50.458294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9fba2-f365-4b4a-a50b-cd7530c88ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:50.459123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9fba2-f365-4b4a-a50b-cd7530c88ef8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:50.461253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 664/1309] 10.222.0.43 () {54 vars in 964 bytes} [Thu Jun 23 11:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:50.473662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e069be-a1d4-4095-96e4-da94c51ae985 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:50.473960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e069be-a1d4-4095-96e4-da94c51ae985 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:50.474818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e069be-a1d4-4095-96e4-da94c51ae985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:50.569311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c9e069be-a1d4-4095-96e4-da94c51ae985 None None] MFA Rules not processed for user `cf08ecfc3fed44e2bc1ccff8521a4ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:50.570569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9e069be-a1d4-4095-96e4-da94c51ae985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:50.594435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:50.657702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 646/1310] 10.222.0.43 () {58 vars in 1061 bytes} [Thu Jun 23 11:01:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:50.669291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9fba2-f365-4b4a-a50b-cd7530c88ef8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:50.671433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-28f9fba2-f365-4b4a-a50b-cd7530c88ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:50.814547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9fba2-f365-4b4a-a50b-cd7530c88ef8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:50.899126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ccd88-bcc2-4a85-93aa-30f8b724edee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:50.899900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ccd88-bcc2-4a85-93aa-30f8b724edee service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:50.900362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ccd88-bcc2-4a85-93aa-30f8b724edee service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:50.900804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ccd88-bcc2-4a85-93aa-30f8b724edee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:50.902827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-879ccd88-bcc2-4a85-93aa-30f8b724edee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:50.966104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:51.024755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ccd88-bcc2-4a85-93aa-30f8b724edee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:51.026212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ccd88-bcc2-4a85-93aa-30f8b724edee service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:51.046803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 665/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:53.374788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc6bf5-2f9c-47a4-9ce3-bcc237f799f9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.375343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc6bf5-2f9c-47a4-9ce3-bcc237f799f9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.375455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc6bf5-2f9c-47a4-9ce3-bcc237f799f9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.376745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 647/1312] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:53.382252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2fdef-25ab-4485-8a24-d2570bb79d8c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.382695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2fdef-25ab-4485-8a24-d2570bb79d8c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.382823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2fdef-25ab-4485-8a24-d2570bb79d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.455578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-11f2fdef-25ab-4485-8a24-d2570bb79d8c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:53.456793 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-11f2fdef-25ab-4485-8a24-d2570bb79d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.561875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 666/1313] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:53.601378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc6bf5-2f9c-47a4-9ce3-bcc237f799f9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:53.602956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ccbc6bf5-2f9c-47a4-9ce3-bcc237f799f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.719331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc6bf5-2f9c-47a4-9ce3-bcc237f799f9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:53.791293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:53.793558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e72a78-7ff2-4574-86b9-cb0958817773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:53.794203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e72a78-7ff2-4574-86b9-cb0958817773 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:53.794491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e72a78-7ff2-4574-86b9-cb0958817773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:53.794768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e72a78-7ff2-4574-86b9-cb0958817773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:53.796491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-61e72a78-7ff2-4574-86b9-cb0958817773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:53.909908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e72a78-7ff2-4574-86b9-cb0958817773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:53.911520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e72a78-7ff2-4574-86b9-cb0958817773 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:53.932513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 648/1314] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:56.469997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185855-3a9a-45e6-98a1-f3688b919f83 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.470526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185855-3a9a-45e6-98a1-f3688b919f83 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.470618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e185855-3a9a-45e6-98a1-f3688b919f83 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.471968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 667/1315] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:56.477134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2905efc-cda0-4328-8a10-96d5f99615bc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.477549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2905efc-cda0-4328-8a10-96d5f99615bc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.477704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2905efc-cda0-4328-8a10-96d5f99615bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.551540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b2905efc-cda0-4328-8a10-96d5f99615bc None None] MFA Rules not processed for user `1f7e314d61674e539a24c3878d778b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:01:56.552742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b2905efc-cda0-4328-8a10-96d5f99615bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:56.607580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:56.653302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 649/1316] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:01:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:01:56.694389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e185855-3a9a-45e6-98a1-f3688b919f83 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:01:56.695961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1e185855-3a9a-45e6-98a1-f3688b919f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:56.705575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:56.800641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e185855-3a9a-45e6-98a1-f3688b919f83 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:01:56.877451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10e4eb-090e-48ff-875a-bcfab4b41046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:01:56.878011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10e4eb-090e-48ff-875a-bcfab4b41046 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:56.878336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10e4eb-090e-48ff-875a-bcfab4b41046 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:56.878615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10e4eb-090e-48ff-875a-bcfab4b41046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:56.880314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3e10e4eb-090e-48ff-875a-bcfab4b41046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:01:56.945348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:01:56.990808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10e4eb-090e-48ff-875a-bcfab4b41046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:01:56.992295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10e4eb-090e-48ff-875a-bcfab4b41046 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:01:57.011377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 668/1317] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:01:59.941550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e84f0-acf4-49d6-a5e4-9b0c36764d46 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:59.942001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e84f0-acf4-49d6-a5e4-9b0c36764d46 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:59.942096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e84f0-acf4-49d6-a5e4-9b0c36764d46 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:01:59.944060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 650/1318] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:01:59.949086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63cc8c-2bb4-49b5-9f2f-8aca1411a1bd None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:01:59.949466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63cc8c-2bb4-49b5-9f2f-8aca1411a1bd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:01:59.949640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63cc8c-2bb4-49b5-9f2f-8aca1411a1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:00.012185 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ee63cc8c-2bb4-49b5-9f2f-8aca1411a1bd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:00.013379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ee63cc8c-2bb4-49b5-9f2f-8aca1411a1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:00.107306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 669/1319] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:01:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:00.146358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e84f0-acf4-49d6-a5e4-9b0c36764d46 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:00.147926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ce7e84f0-acf4-49d6-a5e4-9b0c36764d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:00.248437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e84f0-acf4-49d6-a5e4-9b0c36764d46 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:00.320460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492bb88-72bc-4ee0-b2e6-9669a4c511fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:00.321106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bb88-72bc-4ee0-b2e6-9669a4c511fb service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:00.321395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bb88-72bc-4ee0-b2e6-9669a4c511fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:00.321676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bb88-72bc-4ee0-b2e6-9669a4c511fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:00.323351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c492bb88-72bc-4ee0-b2e6-9669a4c511fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:00.330596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:00.433212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492bb88-72bc-4ee0-b2e6-9669a4c511fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:00.434685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492bb88-72bc-4ee0-b2e6-9669a4c511fb service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:00.452606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 651/1320] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:03.450815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e2049-e42c-44b7-aebd-24971a3e94d0 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.451453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e2049-e42c-44b7-aebd-24971a3e94d0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.451600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e2049-e42c-44b7-aebd-24971a3e94d0 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.452913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 670/1321] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:03.458686 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7cc9-ed2c-42cc-9c2f-23df64b6312f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.459127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7cc9-ed2c-42cc-9c2f-23df64b6312f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.459288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7cc9-ed2c-42cc-9c2f-23df64b6312f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.532453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1a4f7cc9-ed2c-42cc-9c2f-23df64b6312f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:03.533662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1a4f7cc9-ed2c-42cc-9c2f-23df64b6312f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:03.593049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:03.635470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 652/1322] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:03.669922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e2049-e42c-44b7-aebd-24971a3e94d0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:03.671622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8a4e2049-e42c-44b7-aebd-24971a3e94d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:03.694256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:03.796063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e2049-e42c-44b7-aebd-24971a3e94d0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:03.875672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebda410-19e3-45e5-9523-caba839670b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:03.876331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebda410-19e3-45e5-9523-caba839670b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:03.876629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebda410-19e3-45e5-9523-caba839670b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:03.876912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebda410-19e3-45e5-9523-caba839670b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:03.878557 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4ebda410-19e3-45e5-9523-caba839670b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:03.955643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:03.995421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebda410-19e3-45e5-9523-caba839670b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:03.996958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebda410-19e3-45e5-9523-caba839670b8 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:04.016889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 671/1323] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:06.116222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b3d4-7bdb-4440-a6b6-627b9cc2f62f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.116764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b3d4-7bdb-4440-a6b6-627b9cc2f62f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.116856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b3d4-7bdb-4440-a6b6-627b9cc2f62f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.118215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 653/1324] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:06.123430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e41fb-e5c1-48e6-9cb4-d7b25b0a3ad9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.123881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e41fb-e5c1-48e6-9cb4-d7b25b0a3ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.123981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e41fb-e5c1-48e6-9cb4-d7b25b0a3ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.188727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-870e41fb-e5c1-48e6-9cb4-d7b25b0a3ad9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:06.189849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-870e41fb-e5c1-48e6-9cb4-d7b25b0a3ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:06.286310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 672/1325] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:06.324131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b3d4-7bdb-4440-a6b6-627b9cc2f62f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:06.332970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f3b1b3d4-7bdb-4440-a6b6-627b9cc2f62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:06.436083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b3d4-7bdb-4440-a6b6-627b9cc2f62f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:06.510007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08571b0e-de50-4822-83ab-a07e4efa9c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:06.510670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08571b0e-de50-4822-83ab-a07e4efa9c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:06.510956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08571b0e-de50-4822-83ab-a07e4efa9c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:06.511261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08571b0e-de50-4822-83ab-a07e4efa9c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:06.512901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-08571b0e-de50-4822-83ab-a07e4efa9c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:06.530278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:06.628463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08571b0e-de50-4822-83ab-a07e4efa9c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:06.630440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08571b0e-de50-4822-83ab-a07e4efa9c95 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:06.652976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 654/1326] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:08.733995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895baa8-e2dc-449c-ba86-69f2a3a8d276 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.734537 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895baa8-e2dc-449c-ba86-69f2a3a8d276 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.734537 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895baa8-e2dc-449c-ba86-69f2a3a8d276 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:08.735949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 673/1327] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:08.741240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a73b6a-d45b-4a6f-bff1-2f9453704884 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.741673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a73b6a-d45b-4a6f-bff1-2f9453704884 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.741798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a73b6a-d45b-4a6f-bff1-2f9453704884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:08.805989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e0a73b6a-d45b-4a6f-bff1-2f9453704884 None None] MFA Rules not processed for user `1f7e314d61674e539a24c3878d778b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:08.807226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e0a73b6a-d45b-4a6f-bff1-2f9453704884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:08.872578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:08.911342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 655/1328] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:08.961166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c135757-4d2d-4b7e-a821-578efbeb81d6 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.961460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c135757-4d2d-4b7e-a821-578efbeb81d6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.961760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c135757-4d2d-4b7e-a821-578efbeb81d6 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:08.963208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 674/1329] 10.222.0.43 () {54 vars in 964 bytes} [Thu Jun 23 11:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:08.969822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661560-3981-4cde-8e65-2a8127d85fb1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:08.970206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661560-3981-4cde-8e65-2a8127d85fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:08.970554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661560-3981-4cde-8e65-2a8127d85fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.045345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9d661560-3981-4cde-8e65-2a8127d85fb1 None None] MFA Rules not processed for user `cf08ecfc3fed44e2bc1ccff8521a4ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:09.046566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9d661560-3981-4cde-8e65-2a8127d85fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:09.122016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 656/1330] 10.222.0.43 () {58 vars in 1061 bytes} [Thu Jun 23 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:09.131667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c135757-4d2d-4b7e-a821-578efbeb81d6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:09.133179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3c135757-4d2d-4b7e-a821-578efbeb81d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:09.165164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:09.238352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c135757-4d2d-4b7e-a821-578efbeb81d6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:09.310841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28e858-6dfc-41e1-9246-13b196750e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:09.311531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28e858-6dfc-41e1-9246-13b196750e33 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:09.311819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28e858-6dfc-41e1-9246-13b196750e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:09.312099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28e858-6dfc-41e1-9246-13b196750e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:09.313992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4f28e858-6dfc-41e1-9246-13b196750e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:09.413816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:09.450031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28e858-6dfc-41e1-9246-13b196750e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:09.451650 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28e858-6dfc-41e1-9246-13b196750e33 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:09.471377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 675/1331] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:12.357265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595dedf-a8ec-42a1-af17-7be8d8d0dd0c None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.357808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595dedf-a8ec-42a1-af17-7be8d8d0dd0c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.357906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595dedf-a8ec-42a1-af17-7be8d8d0dd0c None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.359240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 657/1332] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:12.364573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83054d6-4c08-45fc-a073-d80d73b3c793 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.365059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83054d6-4c08-45fc-a073-d80d73b3c793 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.365152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83054d6-4c08-45fc-a073-d80d73b3c793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.428189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c83054d6-4c08-45fc-a073-d80d73b3c793 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:12.429381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c83054d6-4c08-45fc-a073-d80d73b3c793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.524514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 676/1333] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:12.559587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595dedf-a8ec-42a1-af17-7be8d8d0dd0c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:12.561160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d595dedf-a8ec-42a1-af17-7be8d8d0dd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.631068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:12.664776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595dedf-a8ec-42a1-af17-7be8d8d0dd0c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:12.745722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137f155-b08d-492e-aee5-9d70db1f04c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:12.746377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137f155-b08d-492e-aee5-9d70db1f04c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:12.746667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137f155-b08d-492e-aee5-9d70db1f04c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:12.746947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137f155-b08d-492e-aee5-9d70db1f04c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:12.748627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9137f155-b08d-492e-aee5-9d70db1f04c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:12.855485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137f155-b08d-492e-aee5-9d70db1f04c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:12.856919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137f155-b08d-492e-aee5-9d70db1f04c0 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:12.867348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:12.875725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 658/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:14.922245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e95fd-c0ec-4744-87b7-cf90b3104f8b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.922672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e95fd-c0ec-4744-87b7-cf90b3104f8b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.922834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e95fd-c0ec-4744-87b7-cf90b3104f8b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.924281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 677/1335] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:14.929633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145164-845e-4498-85f0-bbb1f98c6e15 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:14.930036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145164-845e-4498-85f0-bbb1f98c6e15 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:14.930194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145164-845e-4498-85f0-bbb1f98c6e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:14.995796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7c145164-845e-4498-85f0-bbb1f98c6e15 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:14.997003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7c145164-845e-4498-85f0-bbb1f98c6e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.093644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 659/1336] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:15.133350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e95fd-c0ec-4744-87b7-cf90b3104f8b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:15.134883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c83e95fd-c0ec-4744-87b7-cf90b3104f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.174154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:15.236502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e95fd-c0ec-4744-87b7-cf90b3104f8b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:15.309104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01b533-3af6-4eac-845e-c36962a92dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:15.309744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01b533-3af6-4eac-845e-c36962a92dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:15.310038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01b533-3af6-4eac-845e-c36962a92dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:15.310306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01b533-3af6-4eac-845e-c36962a92dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:15.311963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ba01b533-3af6-4eac-845e-c36962a92dc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:15.393129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:15.418356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01b533-3af6-4eac-845e-c36962a92dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:15.419814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01b533-3af6-4eac-845e-c36962a92dc3 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:15.438699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 678/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:18.006596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee2be1-01b1-4fe0-b9be-49080f85f319 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.007158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee2be1-01b1-4fe0-b9be-49080f85f319 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.007158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee2be1-01b1-4fe0-b9be-49080f85f319 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.008556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 660/1338] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:18.013637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeb03c-bc76-4463-b7e5-95d6ceec21c1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.014053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeb03c-bc76-4463-b7e5-95d6ceec21c1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.014201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeb03c-bc76-4463-b7e5-95d6ceec21c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.080937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e1aeb03c-bc76-4463-b7e5-95d6ceec21c1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:18.082657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e1aeb03c-bc76-4463-b7e5-95d6ceec21c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.177935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 679/1339] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:18.214010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee2be1-01b1-4fe0-b9be-49080f85f319 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:18.215597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-46ee2be1-01b1-4fe0-b9be-49080f85f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.304243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:18.343627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee2be1-01b1-4fe0-b9be-49080f85f319 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:18.435223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721aa805-2837-4030-986d-1ee4057ae67f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:18.435816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721aa805-2837-4030-986d-1ee4057ae67f service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:18.436106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721aa805-2837-4030-986d-1ee4057ae67f service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:18.436385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721aa805-2837-4030-986d-1ee4057ae67f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:18.438344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-721aa805-2837-4030-986d-1ee4057ae67f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:18.545827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721aa805-2837-4030-986d-1ee4057ae67f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:18.547292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721aa805-2837-4030-986d-1ee4057ae67f service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:18.567751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 661/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:22.993873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b64fa-7b9a-4195-afd2-d74587069029 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:22.994372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b64fa-7b9a-4195-afd2-d74587069029 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:22.994372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b64fa-7b9a-4195-afd2-d74587069029 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:22.995952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 680/1341] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:23.001259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c08874-1be3-4f6e-b90f-752bceb117d6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:23.001730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c08874-1be3-4f6e-b90f-752bceb117d6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:23.001834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c08874-1be3-4f6e-b90f-752bceb117d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:23.009624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:23.089792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-32c08874-1be3-4f6e-b90f-752bceb117d6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:23.089792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-32c08874-1be3-4f6e-b90f-752bceb117d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:23.207210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 662/1342] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:23.244260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b64fa-7b9a-4195-afd2-d74587069029 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:23.246456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9e0b64fa-7b9a-4195-afd2-d74587069029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:23.297616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:23.344714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b64fa-7b9a-4195-afd2-d74587069029 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:23.349380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d32b16-f6db-4505-a22b-ec1d4e5af4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:23.349984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d32b16-f6db-4505-a22b-ec1d4e5af4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:23.350272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d32b16-f6db-4505-a22b-ec1d4e5af4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:23.350550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d32b16-f6db-4505-a22b-ec1d4e5af4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:23.360287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d32b16-f6db-4505-a22b-ec1d4e5af4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:23.361728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d32b16-f6db-4505-a22b-ec1d4e5af4d8 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:23.371851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 681/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:26.437670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8fa4e-82aa-453a-be1c-1230075329b0 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.438244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8fa4e-82aa-453a-be1c-1230075329b0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.438244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8fa4e-82aa-453a-be1c-1230075329b0 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.439663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 663/1344] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:26.445945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c1f5a-a240-48d4-bdf6-f2e9be4d0970 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.446515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c1f5a-a240-48d4-bdf6-f2e9be4d0970 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.446515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c1f5a-a240-48d4-bdf6-f2e9be4d0970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.474723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6b4c1f5a-a240-48d4-bdf6-f2e9be4d0970 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:26.475973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6b4c1f5a-a240-48d4-bdf6-f2e9be4d0970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:26.561745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 682/1345] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:26.599920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8fa4e-82aa-453a-be1c-1230075329b0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:26.610344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8fa4e-82aa-453a-be1c-1230075329b0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:26.614304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdd53d-2ff9-4ef3-a092-88b2050f95b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:26.614933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd53d-2ff9-4ef3-a092-88b2050f95b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:26.615238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd53d-2ff9-4ef3-a092-88b2050f95b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:26.615516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdd53d-2ff9-4ef3-a092-88b2050f95b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:26.624473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdd53d-2ff9-4ef3-a092-88b2050f95b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:26.625926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdd53d-2ff9-4ef3-a092-88b2050f95b9 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:26.636316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 664/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:31.118620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f8952-cef4-4e57-b647-caf5d0c9c4f0 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:31.119120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f8952-cef4-4e57-b647-caf5d0c9c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:31.119271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f8952-cef4-4e57-b647-caf5d0c9c4f0 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:31.120584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 683/1347] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:31.125822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee476dbe-b8cf-406e-b0c8-5c7cd6cd91b0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:31.126262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee476dbe-b8cf-406e-b0c8-5c7cd6cd91b0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:31.126455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee476dbe-b8cf-406e-b0c8-5c7cd6cd91b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:31.145112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ee476dbe-b8cf-406e-b0c8-5c7cd6cd91b0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:31.146322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ee476dbe-b8cf-406e-b0c8-5c7cd6cd91b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:31.154264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 665/1348] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:31.196032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f8952-cef4-4e57-b647-caf5d0c9c4f0 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:31.219748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f8952-cef4-4e57-b647-caf5d0c9c4f0 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:31.219748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f247f-b77a-4ccc-968b-1341803d295e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:31.219748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f247f-b77a-4ccc-968b-1341803d295e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:31.219748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f247f-b77a-4ccc-968b-1341803d295e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:31.220109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f247f-b77a-4ccc-968b-1341803d295e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:31.229904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f247f-b77a-4ccc-968b-1341803d295e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:31.231645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f247f-b77a-4ccc-968b-1341803d295e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:31.241297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 684/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:36.194641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665cbb-3306-400e-b444-510e437d096e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.195511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665cbb-3306-400e-b444-510e437d096e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.197571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665cbb-3306-400e-b444-510e437d096e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.199572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 666/1350] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:02:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:36.205333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da80f6-baa0-417d-8a3a-78f116af45f7 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:36.206259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da80f6-baa0-417d-8a3a-78f116af45f7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:36.207006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da80f6-baa0-417d-8a3a-78f116af45f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:36.333796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-09da80f6-baa0-417d-8a3a-78f116af45f7 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:36.335372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-09da80f6-baa0-417d-8a3a-78f116af45f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:36.363679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:36.412466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 685/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:40.762321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5f490-5353-4991-975a-35735d9eee7f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.763574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5f490-5353-4991-975a-35735d9eee7f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.764282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5f490-5353-4991-975a-35735d9eee7f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.766453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 667/1352] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:02:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:40.772909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6602ace-06fa-4120-9848-021a1c98ebb6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:40.773849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6602ace-06fa-4120-9848-021a1c98ebb6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:40.774338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6602ace-06fa-4120-9848-021a1c98ebb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:40.793796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c6602ace-06fa-4120-9848-021a1c98ebb6 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:40.795146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c6602ace-06fa-4120-9848-021a1c98ebb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:40.804155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 686/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:02:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:41.139424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=72 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:42.235719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d92d55-2cc2-474a-b675-d21924312eae None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:42.236338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d92d55-2cc2-474a-b675-d21924312eae None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:42.236338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d92d55-2cc2-474a-b675-d21924312eae None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:42.238285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 668/1354] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:42.243692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b45c5-5af8-4cb0-86e3-9b100c060df9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:42.244147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b45c5-5af8-4cb0-86e3-9b100c060df9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:42.244234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b45c5-5af8-4cb0-86e3-9b100c060df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:42.262730 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0c4b45c5-5af8-4cb0-86e3-9b100c060df9 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:42.263964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0c4b45c5-5af8-4cb0-86e3-9b100c060df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:42.272598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 687/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:02:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:46.075703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71463a3e-6913-4756-adc7-151d56feeb09 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.076562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71463a3e-6913-4756-adc7-151d56feeb09 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.077003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71463a3e-6913-4756-adc7-151d56feeb09 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.083949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 669/1356] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:46.085697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77e1a4-b5b7-4d94-bf62-91b37ae6284b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.086116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77e1a4-b5b7-4d94-bf62-91b37ae6284b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.086321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77e1a4-b5b7-4d94-bf62-91b37ae6284b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.107386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3e77e1a4-b5b7-4d94-bf62-91b37ae6284b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:46.108731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3e77e1a4-b5b7-4d94-bf62-91b37ae6284b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:46.117448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 688/1357] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:46.172587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d29dc-7b6f-4e73-90d3-bb56da1703d1 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.173399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d29dc-7b6f-4e73-90d3-bb56da1703d1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.173942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d29dc-7b6f-4e73-90d3-bb56da1703d1 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.175887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 670/1358] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 11:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:46.185039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b76d5-e731-4b29-a56e-edb895e71462 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.188096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b76d5-e731-4b29-a56e-edb895e71462 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.188096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b76d5-e731-4b29-a56e-edb895e71462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.241106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1a0b76d5-e731-4b29-a56e-edb895e71462 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:46.242342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1a0b76d5-e731-4b29-a56e-edb895e71462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:46.332289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 689/1359] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 11:02:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:46.340208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d29dc-7b6f-4e73-90d3-bb56da1703d1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:46.356446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d29dc-7b6f-4e73-90d3-bb56da1703d1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:46.363045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019b398-9f35-4827-a78d-1bf6da0876be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:46.370723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b398-9f35-4827-a78d-1bf6da0876be service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:46.370723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b398-9f35-4827-a78d-1bf6da0876be service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:46.370723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019b398-9f35-4827-a78d-1bf6da0876be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:46.374016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019b398-9f35-4827-a78d-1bf6da0876be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:46.375722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019b398-9f35-4827-a78d-1bf6da0876be service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:46.386480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 671/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:48.655545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d78a55-4dee-44c9-b06e-8473fdb087bf None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:48.656142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d78a55-4dee-44c9-b06e-8473fdb087bf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:48.656142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d78a55-4dee-44c9-b06e-8473fdb087bf None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:48.657640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 690/1361] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:48.663508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e7425-0527-45f2-a3b5-e989b1beb393 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:48.664304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e7425-0527-45f2-a3b5-e989b1beb393 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:48.664809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e7425-0527-45f2-a3b5-e989b1beb393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:48.682671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-873e7425-0527-45f2-a3b5-e989b1beb393 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:48.683909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-873e7425-0527-45f2-a3b5-e989b1beb393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:48.693192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 672/1362] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:48.738202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d78a55-4dee-44c9-b06e-8473fdb087bf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:48.753617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d78a55-4dee-44c9-b06e-8473fdb087bf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:48.759252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ebe42-4ec8-4e30-a9b2-56dc07a89923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:48.760031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ebe42-4ec8-4e30-a9b2-56dc07a89923 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:48.760421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ebe42-4ec8-4e30-a9b2-56dc07a89923 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:48.760802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ebe42-4ec8-4e30-a9b2-56dc07a89923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:48.770234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ebe42-4ec8-4e30-a9b2-56dc07a89923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:48.772002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ebe42-4ec8-4e30-a9b2-56dc07a89923 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:48.782276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 691/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:51.299635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ece97-f207-4f30-bebb-a417d9c669fe None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.300136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ece97-f207-4f30-bebb-a417d9c669fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.300136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ece97-f207-4f30-bebb-a417d9c669fe None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.301489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 673/1364] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:51.306764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2bbe2-c5b6-433a-87ab-8056ee37931a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.307170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2bbe2-c5b6-433a-87ab-8056ee37931a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.307380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2bbe2-c5b6-433a-87ab-8056ee37931a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.326337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-dcf2bbe2-c5b6-433a-87ab-8056ee37931a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:51.327573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dcf2bbe2-c5b6-433a-87ab-8056ee37931a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:51.335869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 692/1365] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:51.383039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabdfa0-9c28-4a3c-a9c6-2fc143af6a5e None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.383373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabdfa0-9c28-4a3c-a9c6-2fc143af6a5e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.383751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabdfa0-9c28-4a3c-a9c6-2fc143af6a5e None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.385182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 674/1366] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 11:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:51.390389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f881a32-831b-4809-a847-d873d3a3fc98 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.390690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f881a32-831b-4809-a847-d873d3a3fc98 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.392373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f881a32-831b-4809-a847-d873d3a3fc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.409633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9f881a32-831b-4809-a847-d873d3a3fc98 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:51.410825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9f881a32-831b-4809-a847-d873d3a3fc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:51.420234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 693/1367] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:51.428381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabdfa0-9c28-4a3c-a9c6-2fc143af6a5e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:51.442828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabdfa0-9c28-4a3c-a9c6-2fc143af6a5e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:51.448582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256091a-0c6d-4f55-90ab-a6bb186c75b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:51.449284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256091a-0c6d-4f55-90ab-a6bb186c75b0 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:51.449644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256091a-0c6d-4f55-90ab-a6bb186c75b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:51.449927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256091a-0c6d-4f55-90ab-a6bb186c75b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:51.458180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256091a-0c6d-4f55-90ab-a6bb186c75b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:51.459783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256091a-0c6d-4f55-90ab-a6bb186c75b0 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:51.469215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 675/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:53.818309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd9ac1-3a4d-4ac7-97bc-6f5d241cb865 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.818874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd9ac1-3a4d-4ac7-97bc-6f5d241cb865 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.818874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd9ac1-3a4d-4ac7-97bc-6f5d241cb865 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.820266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 694/1369] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:53.825628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20769f4f-152f-4c47-a33c-ba06db1c7e9e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.826043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20769f4f-152f-4c47-a33c-ba06db1c7e9e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.826213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20769f4f-152f-4c47-a33c-ba06db1c7e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.844916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-20769f4f-152f-4c47-a33c-ba06db1c7e9e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:53.846214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-20769f4f-152f-4c47-a33c-ba06db1c7e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:53.855389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 676/1370] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:53.899520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd9ac1-3a4d-4ac7-97bc-6f5d241cb865 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:53.917666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd9ac1-3a4d-4ac7-97bc-6f5d241cb865 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:53.923645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab956c21-bb68-4883-b28f-04c4ee79042d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:53.924409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab956c21-bb68-4883-b28f-04c4ee79042d service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:53.924792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab956c21-bb68-4883-b28f-04c4ee79042d service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:53.925193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab956c21-bb68-4883-b28f-04c4ee79042d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:53.934486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab956c21-bb68-4883-b28f-04c4ee79042d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:53.936495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab956c21-bb68-4883-b28f-04c4ee79042d service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:53.952441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 695/1371] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:54.355637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97560bce-ff3f-497d-9a73-190f08aca52a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:54.355943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97560bce-ff3f-497d-9a73-190f08aca52a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:54.356411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97560bce-ff3f-497d-9a73-190f08aca52a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:54.358067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 677/1372] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:54.363292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b88ab-8d47-4f11-9384-807dba3820a0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:54.363587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b88ab-8d47-4f11-9384-807dba3820a0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:54.363865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b88ab-8d47-4f11-9384-807dba3820a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:54.382865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ea7b88ab-8d47-4f11-9384-807dba3820a0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:54.384190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ea7b88ab-8d47-4f11-9384-807dba3820a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:54.398344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 696/1373] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:54.705985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbabb6-a70b-492c-9723-2d625f9363c4 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:54.706280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbabb6-a70b-492c-9723-2d625f9363c4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:54.706590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbabb6-a70b-492c-9723-2d625f9363c4 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:54.708738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 678/1374] 10.222.0.43 () {54 vars in 963 bytes} [Thu Jun 23 11:02:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:54.713254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c18d92-f61e-43c3-b218-a8237d1bf4d4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:54.713547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c18d92-f61e-43c3-b218-a8237d1bf4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:54.713828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c18d92-f61e-43c3-b218-a8237d1bf4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:54.720784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:02:54.769148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-85c18d92-f61e-43c3-b218-a8237d1bf4d4 None None] MFA Rules not processed for user `2678deaa97fc4012aa0d71f9e505f5ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:54.770513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-85c18d92-f61e-43c3-b218-a8237d1bf4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:54.839327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 697/1375] 10.222.0.43 () {58 vars in 1060 bytes} [Thu Jun 23 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:54.844907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbabb6-a70b-492c-9723-2d625f9363c4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:54.856071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbabb6-a70b-492c-9723-2d625f9363c4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:54.859559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674986ce-151c-4a57-aa55-2728bb0afd68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:54.860205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674986ce-151c-4a57-aa55-2728bb0afd68 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:54.860491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674986ce-151c-4a57-aa55-2728bb0afd68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:54.861580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674986ce-151c-4a57-aa55-2728bb0afd68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:54.871332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674986ce-151c-4a57-aa55-2728bb0afd68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:54.873085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674986ce-151c-4a57-aa55-2728bb0afd68 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:54.884259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 679/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:56.199929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b465-b89a-4979-a9af-8893f48be095 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.200455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b465-b89a-4979-a9af-8893f48be095 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.200455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b465-b89a-4979-a9af-8893f48be095 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.201784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 698/1377] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:56.207742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c56637-189e-421e-afb4-713ea6bd0e75 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.212065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c56637-189e-421e-afb4-713ea6bd0e75 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.212065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c56637-189e-421e-afb4-713ea6bd0e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.226069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-33c56637-189e-421e-afb4-713ea6bd0e75 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:56.227308 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-33c56637-189e-421e-afb4-713ea6bd0e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:56.235134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 680/1378] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:02:56.280068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9b465-b89a-4979-a9af-8893f48be095 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:56.294576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9b465-b89a-4979-a9af-8893f48be095 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:56.300645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6ff29-c000-4e75-bafe-c0e4e9b5fe43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:56.301288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6ff29-c000-4e75-bafe-c0e4e9b5fe43 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:56.301577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6ff29-c000-4e75-bafe-c0e4e9b5fe43 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:56.301859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6ff29-c000-4e75-bafe-c0e4e9b5fe43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:56.310072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6ff29-c000-4e75-bafe-c0e4e9b5fe43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:56.311951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6ff29-c000-4e75-bafe-c0e4e9b5fe43 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:56.322038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 699/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:02:58.549913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd8eb-a1ba-4b31-b4b1-52e992460e50 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.550450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd8eb-a1ba-4b31-b4b1-52e992460e50 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.550547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd8eb-a1ba-4b31-b4b1-52e992460e50 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.551877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 681/1380] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:02:58.557216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146a91b-2b6c-40f5-b5e2-ec5cc7b79bbf None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.557599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146a91b-2b6c-40f5-b5e2-ec5cc7b79bbf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.557774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146a91b-2b6c-40f5-b5e2-ec5cc7b79bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.575639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2146a91b-2b6c-40f5-b5e2-ec5cc7b79bbf None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:02:58.576816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2146a91b-2b6c-40f5-b5e2-ec5cc7b79bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:02:58.589819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 700/1381] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:02:58.639149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dd8eb-a1ba-4b31-b4b1-52e992460e50 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:02:58.662223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dd8eb-a1ba-4b31-b4b1-52e992460e50 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:02:58.669122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b19a3-c7af-45db-89bc-a93248f4c955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:02:58.669966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b19a3-c7af-45db-89bc-a93248f4c955 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:02:58.670407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b19a3-c7af-45db-89bc-a93248f4c955 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:02:58.670858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b19a3-c7af-45db-89bc-a93248f4c955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:02:58.681192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b19a3-c7af-45db-89bc-a93248f4c955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:02:58.683100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b19a3-c7af-45db-89bc-a93248f4c955 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:02:58.696096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 682/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:00.904521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5176-3e5c-44f5-b5a7-956e47e4d14c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:00.905406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5176-3e5c-44f5-b5a7-956e47e4d14c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:00.905838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5176-3e5c-44f5-b5a7-956e47e4d14c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:00.908162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 701/1383] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:03:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:00.913677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6600f-a14f-4dd1-9961-9cd00e24c8e1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:00.914478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6600f-a14f-4dd1-9961-9cd00e24c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:00.914911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6600f-a14f-4dd1-9961-9cd00e24c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:00.933656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0dc6600f-a14f-4dd1-9961-9cd00e24c8e1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:00.934917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0dc6600f-a14f-4dd1-9961-9cd00e24c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:00.943431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 683/1384] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:00.985864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5176-3e5c-44f5-b5a7-956e47e4d14c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:01.000485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5176-3e5c-44f5-b5a7-956e47e4d14c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:01.006075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fa412-904f-4b9f-a0ec-aedbd3a7b61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:01.006725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fa412-904f-4b9f-a0ec-aedbd3a7b61c service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:01.007012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fa412-904f-4b9f-a0ec-aedbd3a7b61c service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:01.007322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fa412-904f-4b9f-a0ec-aedbd3a7b61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:01.016271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fa412-904f-4b9f-a0ec-aedbd3a7b61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:01.018386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fa412-904f-4b9f-a0ec-aedbd3a7b61c service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:01.030844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 702/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:03.165110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffcd2b-a98a-4026-aa7a-9af1de112002 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.165643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffcd2b-a98a-4026-aa7a-9af1de112002 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.165782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffcd2b-a98a-4026-aa7a-9af1de112002 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.167106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 684/1386] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:03.172463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48653383-f0bd-44c9-a5e5-4fbe3ca5db61 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.173648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48653383-f0bd-44c9-a5e5-4fbe3ca5db61 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.173648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48653383-f0bd-44c9-a5e5-4fbe3ca5db61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.192775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-48653383-f0bd-44c9-a5e5-4fbe3ca5db61 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:03.193974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-48653383-f0bd-44c9-a5e5-4fbe3ca5db61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:03.201949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 703/1387] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:03.247081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffcd2b-a98a-4026-aa7a-9af1de112002 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:03.265837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffcd2b-a98a-4026-aa7a-9af1de112002 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:03.274458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a36ebc-b9ee-4a55-ae97-36a110ef43cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:03.275363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a36ebc-b9ee-4a55-ae97-36a110ef43cf service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:03.275735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a36ebc-b9ee-4a55-ae97-36a110ef43cf service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:03.276093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a36ebc-b9ee-4a55-ae97-36a110ef43cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:03.286772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a36ebc-b9ee-4a55-ae97-36a110ef43cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:03.288857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a36ebc-b9ee-4a55-ae97-36a110ef43cf service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:03.301527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 685/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:03:06.016552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5e267-c6e9-4f27-b329-da1024e60088 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:06.017101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5e267-c6e9-4f27-b329-da1024e60088 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:06.017217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5e267-c6e9-4f27-b329-da1024e60088 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:06.018523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 704/1389] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:06.024405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21481-7db2-4aa8-b128-cbb8e7c1cb96 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:06.024828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21481-7db2-4aa8-b128-cbb8e7c1cb96 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:06.025217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21481-7db2-4aa8-b128-cbb8e7c1cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:06.048978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-95d21481-7db2-4aa8-b128-cbb8e7c1cb96 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:06.050635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-95d21481-7db2-4aa8-b128-cbb8e7c1cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:06.061511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 686/1390] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:06.106440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5e267-c6e9-4f27-b329-da1024e60088 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:06.125539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5e267-c6e9-4f27-b329-da1024e60088 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:06.133410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22379c-7efa-4332-8588-8c846e0e6ce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:06.134465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22379c-7efa-4332-8588-8c846e0e6ce0 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:06.134811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22379c-7efa-4332-8588-8c846e0e6ce0 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:06.135278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22379c-7efa-4332-8588-8c846e0e6ce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:06.147677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22379c-7efa-4332-8588-8c846e0e6ce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:06.149789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22379c-7efa-4332-8588-8c846e0e6ce0 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:06.163662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 705/1391] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:08.638703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f844bc-719a-4d2c-a814-235a5efcd3c8 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:08.639283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f844bc-719a-4d2c-a814-235a5efcd3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:08.639401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f844bc-719a-4d2c-a814-235a5efcd3c8 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:08.640742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 687/1392] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:08.646232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780861-df25-46f0-b3f3-bde0a0201b95 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:08.646660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780861-df25-46f0-b3f3-bde0a0201b95 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:08.646916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780861-df25-46f0-b3f3-bde0a0201b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:08.664770 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0e780861-df25-46f0-b3f3-bde0a0201b95 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:08.665970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0e780861-df25-46f0-b3f3-bde0a0201b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:08.674507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 706/1393] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:08.718026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f844bc-719a-4d2c-a814-235a5efcd3c8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:08.734004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f844bc-719a-4d2c-a814-235a5efcd3c8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:08.740761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad76e-2713-45b7-98f7-d61adae15652 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:08.741412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad76e-2713-45b7-98f7-d61adae15652 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:08.741703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad76e-2713-45b7-98f7-d61adae15652 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:08.741988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad76e-2713-45b7-98f7-d61adae15652 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:08.751322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad76e-2713-45b7-98f7-d61adae15652 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:08.752928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad76e-2713-45b7-98f7-d61adae15652 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:08.762520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 688/1394] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:10.483511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb0ad9-a4f2-4014-b32e-a3766e2752e3 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.484080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb0ad9-a4f2-4014-b32e-a3766e2752e3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.484080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb0ad9-a4f2-4014-b32e-a3766e2752e3 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.485532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 707/1395] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:10.490873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474dfdb-b390-4fd7-aed2-7adeb9670590 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.491318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474dfdb-b390-4fd7-aed2-7adeb9670590 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.491462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474dfdb-b390-4fd7-aed2-7adeb9670590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.509596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a474dfdb-b390-4fd7-aed2-7adeb9670590 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:10.510807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a474dfdb-b390-4fd7-aed2-7adeb9670590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:10.519329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 689/1396] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:10.575266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df226bc-6e4b-4bae-9525-64b9e944758b None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.575565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df226bc-6e4b-4bae-9525-64b9e944758b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.575848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df226bc-6e4b-4bae-9525-64b9e944758b None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.577212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 708/1397] 10.222.0.43 () {54 vars in 962 bytes} [Thu Jun 23 11:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:10.582320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0ba4b-1daf-4e86-a4b9-dc64ccdd5a82 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.582618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0ba4b-1daf-4e86-a4b9-dc64ccdd5a82 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.582904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0ba4b-1daf-4e86-a4b9-dc64ccdd5a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.632061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-88f0ba4b-1daf-4e86-a4b9-dc64ccdd5a82 None None] MFA Rules not processed for user `282c8662f2a2431b97d1bf122eabf5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:10.633264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-88f0ba4b-1daf-4e86-a4b9-dc64ccdd5a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:10.694993 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 690/1398] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Jun 23 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:10.702649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df226bc-6e4b-4bae-9525-64b9e944758b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:10.713319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df226bc-6e4b-4bae-9525-64b9e944758b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:10.716813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74a591-18b3-47ef-95a5-897db6158270 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:10.717450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a591-18b3-47ef-95a5-897db6158270 service glance] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.717737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a591-18b3-47ef-95a5-897db6158270 service glance] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.718013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74a591-18b3-47ef-95a5-897db6158270 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.727278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74a591-18b3-47ef-95a5-897db6158270 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:10.728718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74a591-18b3-47ef-95a5-897db6158270 service glance] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:10.738738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 709/1399] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:10.991165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad841575-510d-4d77-82ba-e144b026f81f None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.991410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad841575-510d-4d77-82ba-e144b026f81f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.991723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad841575-510d-4d77-82ba-e144b026f81f None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:10.993095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 691/1400] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:10.998400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a07105-2cb9-45d8-bccf-1bf0dcf25b09 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:10.998761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a07105-2cb9-45d8-bccf-1bf0dcf25b09 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:10.999115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a07105-2cb9-45d8-bccf-1bf0dcf25b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.017724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d9a07105-2cb9-45d8-bccf-1bf0dcf25b09 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:11.018931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d9a07105-2cb9-45d8-bccf-1bf0dcf25b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:11.027405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 710/1401] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:11.071534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad841575-510d-4d77-82ba-e144b026f81f None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:11.086778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad841575-510d-4d77-82ba-e144b026f81f None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:11.092338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096034eb-7472-4c71-aec2-99611ce9ea1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:11.094144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096034eb-7472-4c71-aec2-99611ce9ea1b service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.094144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096034eb-7472-4c71-aec2-99611ce9ea1b service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.094144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096034eb-7472-4c71-aec2-99611ce9ea1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.102118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096034eb-7472-4c71-aec2-99611ce9ea1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:11.103776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096034eb-7472-4c71-aec2-99611ce9ea1b service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:11.113816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 692/1402] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:11.647851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217941-b740-4949-b09f-faa994c15711 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.648310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217941-b740-4949-b09f-faa994c15711 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.648401 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb217941-b740-4949-b09f-faa994c15711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.698206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-eb217941-b740-4949-b09f-faa994c15711 None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:11.699718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-eb217941-b740-4949-b09f-faa994c15711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:11.733376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:11.766369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 711/1403] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:11.809023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b260b1-d894-42a7-ae2f-1ae5cd5258e5 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.809514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b260b1-d894-42a7-ae2f-1ae5cd5258e5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.809762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b260b1-d894-42a7-ae2f-1ae5cd5258e5 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.811509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 693/1404] 10.222.0.43 () {54 vars in 942 bytes} [Thu Jun 23 11:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:11.817949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba167c-bc59-46fa-8109-8f031aa6b005 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:11.818583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba167c-bc59-46fa-8109-8f031aa6b005 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:11.818583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba167c-bc59-46fa-8109-8f031aa6b005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:11.869387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0fba167c-bc59-46fa-8109-8f031aa6b005 None None] MFA Rules not processed for user `f766fee8481a4eccbc56d4af0c2afbf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:11.870590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0fba167c-bc59-46fa-8109-8f031aa6b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:11.951044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 712/1405] 10.222.0.43 () {58 vars in 1039 bytes} [Thu Jun 23 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:11.974106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b260b1-d894-42a7-ae2f-1ae5cd5258e5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:11.983766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:11.993211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b260b1-d894-42a7-ae2f-1ae5cd5258e5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:12.002199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5c4c8-baac-4cda-88a7-49eb8a671313 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:12.003143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5c4c8-baac-4cda-88a7-49eb8a671313 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:12.003737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5c4c8-baac-4cda-88a7-49eb8a671313 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:12.004233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5c4c8-baac-4cda-88a7-49eb8a671313 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:12.014634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5c4c8-baac-4cda-88a7-49eb8a671313 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:12.018237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5c4c8-baac-4cda-88a7-49eb8a671313 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:12.028215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 694/1406] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:03:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:03:13.369926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93086dd-f261-4fc2-ab4b-63134c9fda17 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.370602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93086dd-f261-4fc2-ab4b-63134c9fda17 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.370602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93086dd-f261-4fc2-ab4b-63134c9fda17 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.372291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 713/1407] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:13.377394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390b4e0-5868-4309-a984-87178d689dd1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.377897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390b4e0-5868-4309-a984-87178d689dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.378016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390b4e0-5868-4309-a984-87178d689dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.397387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0390b4e0-5868-4309-a984-87178d689dd1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:13.398597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0390b4e0-5868-4309-a984-87178d689dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:13.415246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 695/1408] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:13.458712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93086dd-f261-4fc2-ab4b-63134c9fda17 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:13.477054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93086dd-f261-4fc2-ab4b-63134c9fda17 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:13.482710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e79afe-cae9-404e-adeb-889881c77d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:13.483389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e79afe-cae9-404e-adeb-889881c77d5a service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:13.483677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e79afe-cae9-404e-adeb-889881c77d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:13.483961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e79afe-cae9-404e-adeb-889881c77d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:13.492891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e79afe-cae9-404e-adeb-889881c77d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:13.494511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e79afe-cae9-404e-adeb-889881c77d5a service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:13.505065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 714/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:23.392523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=8 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:23.500605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=62 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:03:35.976805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed950c2-36d5-47fa-b774-811a335a4ce7 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:35.977226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed950c2-36d5-47fa-b774-811a335a4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:35.977360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed950c2-36d5-47fa-b774-811a335a4ce7 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:35.978738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 696/1410] 10.222.0.43 () {54 vars in 918 bytes} [Thu Jun 23 11:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:35.984032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8cef0-c76e-4dfc-abf5-e99b2474c327 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:35.984493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8cef0-c76e-4dfc-abf5-e99b2474c327 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:35.984646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8cef0-c76e-4dfc-abf5-e99b2474c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:36.003204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-83e8cef0-c76e-4dfc-abf5-e99b2474c327 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:36.004428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-83e8cef0-c76e-4dfc-abf5-e99b2474c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:36.013149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 715/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Jun 23 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:36.028043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e0f3f-953f-45d6-abb6-4ef06fe438eb None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:36.028329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e0f3f-953f-45d6-abb6-4ef06fe438eb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:36.028628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e0f3f-953f-45d6-abb6-4ef06fe438eb None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:36.029958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 697/1412] 10.222.0.43 () {54 vars in 918 bytes} [Thu Jun 23 11:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:36.035376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbdc0f-ff12-4239-836d-40a689bf2983 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:36.035706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbdc0f-ff12-4239-836d-40a689bf2983 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:36.035995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbdc0f-ff12-4239-836d-40a689bf2983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:36.058235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-78bbdc0f-ff12-4239-836d-40a689bf2983 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:36.059722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-78bbdc0f-ff12-4239-836d-40a689bf2983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:36.069300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 716/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Jun 23 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:38.665699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7af7c3-1bee-4d6d-8f10-1a91879e812b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:38.666282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7af7c3-1bee-4d6d-8f10-1a91879e812b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:38.666282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7af7c3-1bee-4d6d-8f10-1a91879e812b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:38.667651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:38.673417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadacee0-05d0-4ba7-b9ab-4fc999b0ec41 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:38.673847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadacee0-05d0-4ba7-b9ab-4fc999b0ec41 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:38.673967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadacee0-05d0-4ba7-b9ab-4fc999b0ec41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:38.691553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-eadacee0-05d0-4ba7-b9ab-4fc999b0ec41 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:38.692672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-eadacee0-05d0-4ba7-b9ab-4fc999b0ec41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:38.700334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:38.744473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7af7c3-1bee-4d6d-8f10-1a91879e812b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:38.758676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7af7c3-1bee-4d6d-8f10-1a91879e812b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:38.764008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f67d93-e645-417c-9fab-609cb454a7eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:38.765093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f67d93-e645-417c-9fab-609cb454a7eb service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:38.765382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f67d93-e645-417c-9fab-609cb454a7eb service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:38.765663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f67d93-e645-417c-9fab-609cb454a7eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:38.773815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f67d93-e645-417c-9fab-609cb454a7eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:38.775424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f67d93-e645-417c-9fab-609cb454a7eb service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:38.785490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 699/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:39.688684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fba1c-dfef-4dd1-948f-1ecf4ecf0d67 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.689265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fba1c-dfef-4dd1-948f-1ecf4ecf0d67 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.689265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fba1c-dfef-4dd1-948f-1ecf4ecf0d67 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.690637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:39.696948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679756c1-5299-4bc1-94b8-4904026d724d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.697404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679756c1-5299-4bc1-94b8-4904026d724d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.697695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679756c1-5299-4bc1-94b8-4904026d724d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.716968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-679756c1-5299-4bc1-94b8-4904026d724d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:39.718533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-679756c1-5299-4bc1-94b8-4904026d724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:39.726667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:39.876258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fba1c-dfef-4dd1-948f-1ecf4ecf0d67 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:39.890383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fba1c-dfef-4dd1-948f-1ecf4ecf0d67 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:39.903565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f430-4d6a-4f1b-8c1f-d16c0ead6eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:39.903565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f430-4d6a-4f1b-8c1f-d16c0ead6eb4 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:39.903565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f430-4d6a-4f1b-8c1f-d16c0ead6eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:39.903565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f430-4d6a-4f1b-8c1f-d16c0ead6eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:39.905134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f430-4d6a-4f1b-8c1f-d16c0ead6eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:39.906714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f430-4d6a-4f1b-8c1f-d16c0ead6eb4 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:39.916211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 719/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:40.825595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9ceb6-7499-423a-a2e8-c1c83896b51b None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:40.826164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9ceb6-7499-423a-a2e8-c1c83896b51b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:40.826164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9ceb6-7499-423a-a2e8-c1c83896b51b None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:40.827792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:40.832683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301938b6-c02a-4cf4-9ec3-92a9170d1336 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:40.833117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301938b6-c02a-4cf4-9ec3-92a9170d1336 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:40.833253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301938b6-c02a-4cf4-9ec3-92a9170d1336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:40.851591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-301938b6-c02a-4cf4-9ec3-92a9170d1336 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:40.853017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-301938b6-c02a-4cf4-9ec3-92a9170d1336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:40.861588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:41.011893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9ceb6-7499-423a-a2e8-c1c83896b51b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:41.027495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9ceb6-7499-423a-a2e8-c1c83896b51b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:41.033423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163e3ff-6d15-471f-919e-907494e58082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:41.034090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163e3ff-6d15-471f-919e-907494e58082 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:41.034375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163e3ff-6d15-471f-919e-907494e58082 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:41.034652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163e3ff-6d15-471f-919e-907494e58082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:41.043327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163e3ff-6d15-471f-919e-907494e58082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:41.044916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163e3ff-6d15-471f-919e-907494e58082 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:41.055335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 702/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:41.924196 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dcda3-4611-4b79-9d7d-bbfdce30523c None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:41.924697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dcda3-4611-4b79-9d7d-bbfdce30523c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:41.924795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dcda3-4611-4b79-9d7d-bbfdce30523c None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:41.926140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:41.931111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a452-1779-4db6-93df-063cc090a63b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:41.931546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a452-1779-4db6-93df-063cc090a63b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:41.931713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a452-1779-4db6-93df-063cc090a63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:41.950301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f001a452-1779-4db6-93df-063cc090a63b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:41.952085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f001a452-1779-4db6-93df-063cc090a63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:41.960484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:42.121500 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dcda3-4611-4b79-9d7d-bbfdce30523c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:42.136285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dcda3-4611-4b79-9d7d-bbfdce30523c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:42.142253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fed5c-6228-4487-849e-4a4696f32dbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:42.142893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fed5c-6228-4487-849e-4a4696f32dbd service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:42.143199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fed5c-6228-4487-849e-4a4696f32dbd service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:42.143488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fed5c-6228-4487-849e-4a4696f32dbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:42.151429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fed5c-6228-4487-849e-4a4696f32dbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:42.153013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fed5c-6228-4487-849e-4a4696f32dbd service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:42.162965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 722/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:43.020850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8e524-03af-4840-a21d-f0e2601ec2e9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:43.021431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8e524-03af-4840-a21d-f0e2601ec2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:43.021431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8e524-03af-4840-a21d-f0e2601ec2e9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:43.022765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:03:43.027627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c01f9a-183f-4985-9d18-dea41efba329 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:43.028014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c01f9a-183f-4985-9d18-dea41efba329 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:43.028201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c01f9a-183f-4985-9d18-dea41efba329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:43.046442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c4c01f9a-183f-4985-9d18-dea41efba329 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:43.047663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c4c01f9a-183f-4985-9d18-dea41efba329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:43.055391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:43.204024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8e524-03af-4840-a21d-f0e2601ec2e9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:43.218673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8e524-03af-4840-a21d-f0e2601ec2e9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:43.224031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79a26d-de48-47df-a716-bd7d4a1c917f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:43.224661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79a26d-de48-47df-a716-bd7d4a1c917f service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:43.224947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79a26d-de48-47df-a716-bd7d4a1c917f service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:43.225227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79a26d-de48-47df-a716-bd7d4a1c917f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:43.233831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79a26d-de48-47df-a716-bd7d4a1c917f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:43.235429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79a26d-de48-47df-a716-bd7d4a1c917f service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:43.244601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 705/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:44.136016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c7d5-743e-4a72-9b88-7a7d1133db52 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:44.136651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c7d5-743e-4a72-9b88-7a7d1133db52 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:44.136651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8c7d5-743e-4a72-9b88-7a7d1133db52 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:44.137952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:44.143630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1831-6494-4800-a740-e6009450bd52 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:44.144074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1831-6494-4800-a740-e6009450bd52 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:44.144186 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1831-6494-4800-a740-e6009450bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:44.161945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-092d1831-6494-4800-a740-e6009450bd52 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:44.163140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-092d1831-6494-4800-a740-e6009450bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:44.171066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:44.322244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8c7d5-743e-4a72-9b88-7a7d1133db52 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:44.336916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8c7d5-743e-4a72-9b88-7a7d1133db52 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:44.342839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a47aeb-5908-423f-bf06-f2b347c58dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:44.343502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a47aeb-5908-423f-bf06-f2b347c58dd9 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:44.343787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a47aeb-5908-423f-bf06-f2b347c58dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:44.344082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a47aeb-5908-423f-bf06-f2b347c58dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:44.352038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a47aeb-5908-423f-bf06-f2b347c58dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:44.353606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a47aeb-5908-423f-bf06-f2b347c58dd9 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:44.362794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 725/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:45.226763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c651cd1-58dd-4b17-8193-68cd73252d98 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:45.227332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c651cd1-58dd-4b17-8193-68cd73252d98 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:45.227332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c651cd1-58dd-4b17-8193-68cd73252d98 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:45.228654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:45.234273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120b7b3-b4d8-40e6-a675-ff3ea8a316dc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:45.234689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120b7b3-b4d8-40e6-a675-ff3ea8a316dc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:45.234833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120b7b3-b4d8-40e6-a675-ff3ea8a316dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:45.253032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d120b7b3-b4d8-40e6-a675-ff3ea8a316dc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:45.254224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d120b7b3-b4d8-40e6-a675-ff3ea8a316dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:45.261975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:45.413347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c651cd1-58dd-4b17-8193-68cd73252d98 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:45.428347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c651cd1-58dd-4b17-8193-68cd73252d98 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:45.433866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd9b30-f8f6-48fd-9983-ceb7bdab5553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:45.434503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9b30-f8f6-48fd-9983-ceb7bdab5553 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:45.434786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9b30-f8f6-48fd-9983-ceb7bdab5553 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:45.435065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd9b30-f8f6-48fd-9983-ceb7bdab5553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:45.444503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd9b30-f8f6-48fd-9983-ceb7bdab5553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:45.445309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd9b30-f8f6-48fd-9983-ceb7bdab5553 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:45.455746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 708/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:46.324523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cb53a-1bed-4d10-a76d-70d1c25267f8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:46.325077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cb53a-1bed-4d10-a76d-70d1c25267f8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:46.325188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cb53a-1bed-4d10-a76d-70d1c25267f8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:46.327159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:46.332219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d209ff7-0c9f-4192-9a0b-711895fd6ab4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:46.332678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d209ff7-0c9f-4192-9a0b-711895fd6ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:46.332797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d209ff7-0c9f-4192-9a0b-711895fd6ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:46.350748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4d209ff7-0c9f-4192-9a0b-711895fd6ab4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:46.351983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4d209ff7-0c9f-4192-9a0b-711895fd6ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:46.359956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:46.509779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cb53a-1bed-4d10-a76d-70d1c25267f8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:46.524558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cb53a-1bed-4d10-a76d-70d1c25267f8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:46.529982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1830c-2847-4a87-b2b6-4401fd59d099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:46.530621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1830c-2847-4a87-b2b6-4401fd59d099 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:46.530909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1830c-2847-4a87-b2b6-4401fd59d099 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:46.531214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1830c-2847-4a87-b2b6-4401fd59d099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:46.539247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1830c-2847-4a87-b2b6-4401fd59d099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:46.540826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1830c-2847-4a87-b2b6-4401fd59d099 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:46.551042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 728/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:47.426706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ca435-e6ad-43c6-b30e-5ed1e36546b4 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:47.427341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ca435-e6ad-43c6-b30e-5ed1e36546b4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:47.427341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ca435-e6ad-43c6-b30e-5ed1e36546b4 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:47.428710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:47.433910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c00ec-d505-441d-b6c7-f9907e623569 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:47.434374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c00ec-d505-441d-b6c7-f9907e623569 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:47.434510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c00ec-d505-441d-b6c7-f9907e623569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:47.453047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-519c00ec-d505-441d-b6c7-f9907e623569 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:47.454246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-519c00ec-d505-441d-b6c7-f9907e623569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:47.462160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:47.616716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ca435-e6ad-43c6-b30e-5ed1e36546b4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:47.631114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ca435-e6ad-43c6-b30e-5ed1e36546b4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:47.636516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb9351-3363-4d90-a8c9-ae0d8f9a924b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:47.637149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9351-3363-4d90-a8c9-ae0d8f9a924b service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:47.637434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9351-3363-4d90-a8c9-ae0d8f9a924b service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:47.637714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9351-3363-4d90-a8c9-ae0d8f9a924b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:47.646247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb9351-3363-4d90-a8c9-ae0d8f9a924b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:47.647859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb9351-3363-4d90-a8c9-ae0d8f9a924b service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:47.657722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 711/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:49.591239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098524f-ed01-4363-9f1d-90957e933254 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:49.591778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098524f-ed01-4363-9f1d-90957e933254 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:49.591900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098524f-ed01-4363-9f1d-90957e933254 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:49.593387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:49.598765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5930-3bb2-4b01-989a-18a8ad8e164f None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:49.599201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5930-3bb2-4b01-989a-18a8ad8e164f None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:49.599360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec5930-3bb2-4b01-989a-18a8ad8e164f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:49.617400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-44ec5930-3bb2-4b01-989a-18a8ad8e164f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:49.618589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-44ec5930-3bb2-4b01-989a-18a8ad8e164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:49.627014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:49.664803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098524f-ed01-4363-9f1d-90957e933254 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:49.675933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098524f-ed01-4363-9f1d-90957e933254 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:49.679419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053142e9-4834-42b1-88eb-53b2f89ab931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:49.680054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053142e9-4834-42b1-88eb-53b2f89ab931 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:49.680340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053142e9-4834-42b1-88eb-53b2f89ab931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:49.680620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053142e9-4834-42b1-88eb-53b2f89ab931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:49.690288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053142e9-4834-42b1-88eb-53b2f89ab931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:49.691771 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053142e9-4834-42b1-88eb-53b2f89ab931 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:49.701682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 731/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:51.857749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e848e8-c700-4b80-8743-1f4f7a39e475 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:51.858327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e848e8-c700-4b80-8743-1f4f7a39e475 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:51.858327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e848e8-c700-4b80-8743-1f4f7a39e475 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:51.859733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:51.865174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e50ab-5798-4ac9-be14-85660baee4dc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:51.865601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e50ab-5798-4ac9-be14-85660baee4dc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:51.865735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e50ab-5798-4ac9-be14-85660baee4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:51.883131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-df6e50ab-5798-4ac9-be14-85660baee4dc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:51.884343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-df6e50ab-5798-4ac9-be14-85660baee4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:51.892941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:51.930986 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e848e8-c700-4b80-8743-1f4f7a39e475 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:51.942168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e848e8-c700-4b80-8743-1f4f7a39e475 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:51.945728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015df78-96de-4710-8b22-27b03658fc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:51.946870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015df78-96de-4710-8b22-27b03658fc54 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:51.947158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015df78-96de-4710-8b22-27b03658fc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:51.947473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015df78-96de-4710-8b22-27b03658fc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:51.957432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015df78-96de-4710-8b22-27b03658fc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:51.958919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015df78-96de-4710-8b22-27b03658fc54 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:51.970216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 714/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:54.353621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ef7b-d0b1-4871-81bf-d5685335f897 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:54.354175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ef7b-d0b1-4871-81bf-d5685335f897 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:54.354175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ef7b-d0b1-4871-81bf-d5685335f897 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:54.355631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:03:54.360913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a18e5-dae6-4eb0-a788-5f78ee80bfe2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:54.361356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a18e5-dae6-4eb0-a788-5f78ee80bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:54.361470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a18e5-dae6-4eb0-a788-5f78ee80bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:54.378632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ef0a18e5-dae6-4eb0-a788-5f78ee80bfe2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:54.379848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ef0a18e5-dae6-4eb0-a788-5f78ee80bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:54.387469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:03:54.434168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891ef7b-d0b1-4871-81bf-d5685335f897 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:54.445918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891ef7b-d0b1-4871-81bf-d5685335f897 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:54.449364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc5f27-bae6-4c5b-a17b-1d4b56de5d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:54.449996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc5f27-bae6-4c5b-a17b-1d4b56de5d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:54.450278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc5f27-bae6-4c5b-a17b-1d4b56de5d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:54.450571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc5f27-bae6-4c5b-a17b-1d4b56de5d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:54.462361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc5f27-bae6-4c5b-a17b-1d4b56de5d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:54.463823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc5f27-bae6-4c5b-a17b-1d4b56de5d82 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:54.473228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 734/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:56.644746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f3ba9-7f44-40fd-8f4f-3af9dfb6c4e0 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:56.645284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f3ba9-7f44-40fd-8f4f-3af9dfb6c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:56.645284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f3ba9-7f44-40fd-8f4f-3af9dfb6c4e0 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:56.646661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:56.652016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83907c-5749-4ebe-8c5f-48221299ab85 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:56.652467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83907c-5749-4ebe-8c5f-48221299ab85 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:56.652574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83907c-5749-4ebe-8c5f-48221299ab85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:56.670751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6a83907c-5749-4ebe-8c5f-48221299ab85 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:56.671982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6a83907c-5749-4ebe-8c5f-48221299ab85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:56.679834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:56.719666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f3ba9-7f44-40fd-8f4f-3af9dfb6c4e0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:56.730492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f3ba9-7f44-40fd-8f4f-3af9dfb6c4e0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:56.734345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926ebb8-2134-4741-8b37-d8f20073a4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:56.734989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926ebb8-2134-4741-8b37-d8f20073a4dd service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:56.735298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926ebb8-2134-4741-8b37-d8f20073a4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:56.735578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926ebb8-2134-4741-8b37-d8f20073a4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:56.744605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926ebb8-2134-4741-8b37-d8f20073a4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:56.746026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926ebb8-2134-4741-8b37-d8f20073a4dd service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:56.756119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 717/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:03:58.850311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb83bac-0f89-4bcb-93ec-d68c6b73d510 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:58.850886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb83bac-0f89-4bcb-93ec-d68c6b73d510 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:58.850886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb83bac-0f89-4bcb-93ec-d68c6b73d510 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:58.852308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:03:58.858358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f807d-e292-446a-95ab-234b2d74d1ac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:58.858814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f807d-e292-446a-95ab-234b2d74d1ac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:58.858956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f807d-e292-446a-95ab-234b2d74d1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:58.877076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e40f807d-e292-446a-95ab-234b2d74d1ac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:03:58.878269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e40f807d-e292-446a-95ab-234b2d74d1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:03:58.886002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:03:58.924447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb83bac-0f89-4bcb-93ec-d68c6b73d510 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:03:58.935394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb83bac-0f89-4bcb-93ec-d68c6b73d510 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:03:58.938858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de90c7-9509-4345-a5d1-7a5daac1805e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:03:58.939524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de90c7-9509-4345-a5d1-7a5daac1805e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:03:58.939812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de90c7-9509-4345-a5d1-7a5daac1805e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:03:58.940093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de90c7-9509-4345-a5d1-7a5daac1805e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:03:58.948821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de90c7-9509-4345-a5d1-7a5daac1805e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:03:58.950244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de90c7-9509-4345-a5d1-7a5daac1805e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:03:58.959693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 737/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:01.027958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b30e98-b686-4827-b733-cfd91d511878 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:01.029039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b30e98-b686-4827-b733-cfd91d511878 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:01.029139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b30e98-b686-4827-b733-cfd91d511878 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:01.030514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:01.035829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e55718-6165-4de7-80db-3c8f32f8b3b9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:01.036267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e55718-6165-4de7-80db-3c8f32f8b3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:01.036404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e55718-6165-4de7-80db-3c8f32f8b3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:01.053895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b4e55718-6165-4de7-80db-3c8f32f8b3b9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:01.055082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4e55718-6165-4de7-80db-3c8f32f8b3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:01.063393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:01.101319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b30e98-b686-4827-b733-cfd91d511878 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:01.111846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b30e98-b686-4827-b733-cfd91d511878 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:01.115409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a884-e9ba-45b6-9209-5e5ed13de2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:01.116023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a884-e9ba-45b6-9209-5e5ed13de2de service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:01.116305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a884-e9ba-45b6-9209-5e5ed13de2de service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:01.116581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a884-e9ba-45b6-9209-5e5ed13de2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:01.125461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8a884-e9ba-45b6-9209-5e5ed13de2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:01.126876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8a884-e9ba-45b6-9209-5e5ed13de2de service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:01.136056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 720/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:03.313003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae653d-4b29-48a6-ba6e-e68304477fdc None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:03.313543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae653d-4b29-48a6-ba6e-e68304477fdc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:03.313543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae653d-4b29-48a6-ba6e-e68304477fdc None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:03.314869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:03.320147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4aed-590a-4d8d-bd75-70180d923acd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:03.320573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4aed-590a-4d8d-bd75-70180d923acd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:03.320720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4aed-590a-4d8d-bd75-70180d923acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:03.338576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f44b4aed-590a-4d8d-bd75-70180d923acd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:03.339799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f44b4aed-590a-4d8d-bd75-70180d923acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:03.348222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:03.385847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae653d-4b29-48a6-ba6e-e68304477fdc None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:03.388065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2dae653d-4b29-48a6-ba6e-e68304477fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:03.400789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae653d-4b29-48a6-ba6e-e68304477fdc None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:03.405622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1f5f3-92f4-46e5-b9bb-caca8006f8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:03.406278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1f5f3-92f4-46e5-b9bb-caca8006f8ee service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:03.406567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1f5f3-92f4-46e5-b9bb-caca8006f8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:03.406847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1f5f3-92f4-46e5-b9bb-caca8006f8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:03.416172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1f5f3-92f4-46e5-b9bb-caca8006f8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:03.417633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1f5f3-92f4-46e5-b9bb-caca8006f8ee service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:03.427112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 740/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:05.517323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2173dc-a998-4c83-a170-8920aed56adc None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:05.517891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2173dc-a998-4c83-a170-8920aed56adc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:05.517891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2173dc-a998-4c83-a170-8920aed56adc None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:05.519314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:05.524710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b09fe9-bc1a-4cc2-9d87-6c6ea5ae3319 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:05.525116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b09fe9-bc1a-4cc2-9d87-6c6ea5ae3319 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:05.525311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b09fe9-bc1a-4cc2-9d87-6c6ea5ae3319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:05.532095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=79 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:05.543942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-67b09fe9-bc1a-4cc2-9d87-6c6ea5ae3319 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:05.556578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-67b09fe9-bc1a-4cc2-9d87-6c6ea5ae3319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:05.556578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:05.591822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2173dc-a998-4c83-a170-8920aed56adc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:05.597134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=73 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:05.602691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2173dc-a998-4c83-a170-8920aed56adc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:05.606142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e636ad-21c0-4923-8ff3-f1e05ab623af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:05.613254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e636ad-21c0-4923-8ff3-f1e05ab623af service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:05.613254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e636ad-21c0-4923-8ff3-f1e05ab623af service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:05.613254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e636ad-21c0-4923-8ff3-f1e05ab623af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:05.616078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e636ad-21c0-4923-8ff3-f1e05ab623af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:05.617498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e636ad-21c0-4923-8ff3-f1e05ab623af service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:05.626716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 723/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:07.663445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185213-4356-4efe-b670-15b4c73aaf2d None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:07.664033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185213-4356-4efe-b670-15b4c73aaf2d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:07.664033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185213-4356-4efe-b670-15b4c73aaf2d None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:07.665407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:07.670639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03a6b4-ac08-4033-a834-a22fce5f1ba2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:07.671068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03a6b4-ac08-4033-a834-a22fce5f1ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:07.671238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03a6b4-ac08-4033-a834-a22fce5f1ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:07.689259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1f03a6b4-ac08-4033-a834-a22fce5f1ba2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:07.690473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1f03a6b4-ac08-4033-a834-a22fce5f1ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:07.698380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:07.735736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c185213-4356-4efe-b670-15b4c73aaf2d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:07.746658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c185213-4356-4efe-b670-15b4c73aaf2d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:07.754113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feef8dd-23cb-4719-97fe-bcd7146746aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:07.754113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feef8dd-23cb-4719-97fe-bcd7146746aa service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:07.754113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feef8dd-23cb-4719-97fe-bcd7146746aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:07.754113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feef8dd-23cb-4719-97fe-bcd7146746aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:07.760590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feef8dd-23cb-4719-97fe-bcd7146746aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:07.762090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feef8dd-23cb-4719-97fe-bcd7146746aa service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:07.772278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 743/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:09.862266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7e92-027b-4e3f-bd77-0a2ec67ccb32 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:09.863064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7e92-027b-4e3f-bd77-0a2ec67ccb32 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:09.863064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7e92-027b-4e3f-bd77-0a2ec67ccb32 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:09.864230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:09.869499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae082de-d630-4f36-9e1e-418a6c9a4d47 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:09.869911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae082de-d630-4f36-9e1e-418a6c9a4d47 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:09.870057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae082de-d630-4f36-9e1e-418a6c9a4d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:09.892959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1ae082de-d630-4f36-9e1e-418a6c9a4d47 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:09.894151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1ae082de-d630-4f36-9e1e-418a6c9a4d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:09.908344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:09.945299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d7e92-027b-4e3f-bd77-0a2ec67ccb32 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:09.956670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d7e92-027b-4e3f-bd77-0a2ec67ccb32 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:09.960203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a867e4-1288-4870-b402-3f19a2155cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:09.960846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a867e4-1288-4870-b402-3f19a2155cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:09.961138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a867e4-1288-4870-b402-3f19a2155cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:09.961418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a867e4-1288-4870-b402-3f19a2155cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:09.970639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a867e4-1288-4870-b402-3f19a2155cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:09.972117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a867e4-1288-4870-b402-3f19a2155cd7 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:09.982387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 726/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:19.883025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=11 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:19.977292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=14 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:29.991203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb113e-60f1-4126-a68f-50cfcebf3a38 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:29.991781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb113e-60f1-4126-a68f-50cfcebf3a38 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:29.991781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb113e-60f1-4126-a68f-50cfcebf3a38 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:29.993219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 745/1471] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:04:29.999282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf8966-bf57-4581-a188-7889e9451a8c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:29.999709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf8966-bf57-4581-a188-7889e9451a8c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:29.999863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf8966-bf57-4581-a188-7889e9451a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:30.017850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7daf8966-bf57-4581-a188-7889e9451a8c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:30.019071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7daf8966-bf57-4581-a188-7889e9451a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:30.026929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 727/1472] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:30.075018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221db6-b1cf-4c4e-a3ce-674b3d4e8b42 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:30.075410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221db6-b1cf-4c4e-a3ce-674b3d4e8b42 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:30.075694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91221db6-b1cf-4c4e-a3ce-674b3d4e8b42 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:30.077045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 746/1473] 10.222.0.43 () {54 vars in 962 bytes} [Thu Jun 23 11:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:30.082047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106bd9c-ed89-4c37-a70a-c4e812cd74f3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:30.082342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106bd9c-ed89-4c37-a70a-c4e812cd74f3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:30.082627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106bd9c-ed89-4c37-a70a-c4e812cd74f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:30.101183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6106bd9c-ed89-4c37-a70a-c4e812cd74f3 None None] MFA Rules not processed for user `282c8662f2a2431b97d1bf122eabf5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:30.102379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6106bd9c-ed89-4c37-a70a-c4e812cd74f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:30.109436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 728/1474] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Jun 23 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:30.116447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91221db6-b1cf-4c4e-a3ce-674b3d4e8b42 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:30.126911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91221db6-b1cf-4c4e-a3ce-674b3d4e8b42 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:30.130394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e08abe-1be2-4b9e-b76d-f8f436ed38e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:30.131013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e08abe-1be2-4b9e-b76d-f8f436ed38e3 service glance] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:30.131333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e08abe-1be2-4b9e-b76d-f8f436ed38e3 service glance] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:30.131606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e08abe-1be2-4b9e-b76d-f8f436ed38e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:30.140971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e08abe-1be2-4b9e-b76d-f8f436ed38e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:30.142396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e08abe-1be2-4b9e-b76d-f8f436ed38e3 service glance] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:30.152361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 747/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:32.269847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c1f42-43f5-45dd-8b38-d6e30cfae68d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:32.270365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c1f42-43f5-45dd-8b38-d6e30cfae68d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:32.270365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c1f42-43f5-45dd-8b38-d6e30cfae68d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:32.271729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 729/1476] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:32.277119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931319b1-e60a-4bd7-adc4-d67f357ea8fc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:32.277591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931319b1-e60a-4bd7-adc4-d67f357ea8fc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:32.277691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931319b1-e60a-4bd7-adc4-d67f357ea8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:32.295645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-931319b1-e60a-4bd7-adc4-d67f357ea8fc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:32.296843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-931319b1-e60a-4bd7-adc4-d67f357ea8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:32.304614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 748/1477] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:32.328330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c1f42-43f5-45dd-8b38-d6e30cfae68d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:32.340703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c1f42-43f5-45dd-8b38-d6e30cfae68d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:32.345295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ed45f-238b-419c-ba3b-ace6b42d5310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:32.345942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ed45f-238b-419c-ba3b-ace6b42d5310 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:32.346228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ed45f-238b-419c-ba3b-ace6b42d5310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:32.346505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ed45f-238b-419c-ba3b-ace6b42d5310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:32.347275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ed45f-238b-419c-ba3b-ace6b42d5310 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:32.355027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-7f5ed45f-238b-419c-ba3b-ace6b42d5310 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:04:32.355027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:04:32.355027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:04:32.355027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ed45f-238b-419c-ba3b-ace6b42d5310 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:32.369589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 730/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 11:04:32 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:04:34.417681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cce70-42a0-4b37-9a8b-b4fe8a44cdcf None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:34.418374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cce70-42a0-4b37-9a8b-b4fe8a44cdcf None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:34.418374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cce70-42a0-4b37-9a8b-b4fe8a44cdcf None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:34.419653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 749/1479] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:04:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:34.424891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccea419-68db-45d3-bd75-19d4e821f2ac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:34.425342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccea419-68db-45d3-bd75-19d4e821f2ac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:34.425486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccea419-68db-45d3-bd75-19d4e821f2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:34.443779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cccea419-68db-45d3-bd75-19d4e821f2ac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:34.444980 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cccea419-68db-45d3-bd75-19d4e821f2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:34.452839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 731/1480] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:34.496430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cce70-42a0-4b37-9a8b-b4fe8a44cdcf None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:34.510542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cce70-42a0-4b37-9a8b-b4fe8a44cdcf None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:34.516053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0fd5b-2733-4021-9ad9-59c9eba2d219 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:34.516692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fd5b-2733-4021-9ad9-59c9eba2d219 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:34.516976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fd5b-2733-4021-9ad9-59c9eba2d219 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:34.517257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fd5b-2733-4021-9ad9-59c9eba2d219 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:34.525429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0fd5b-2733-4021-9ad9-59c9eba2d219 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:34.527574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0fd5b-2733-4021-9ad9-59c9eba2d219 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:34.536873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 750/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:36.743937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e4bb-b4a9-4e11-83f6-ad04f87b75ec None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:36.744505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e4bb-b4a9-4e11-83f6-ad04f87b75ec None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:36.744505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e4bb-b4a9-4e11-83f6-ad04f87b75ec None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:36.746409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 732/1482] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:36.751691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897b396-ecde-4137-89e9-db69b7897515 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:36.752087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897b396-ecde-4137-89e9-db69b7897515 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:36.752257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897b396-ecde-4137-89e9-db69b7897515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:36.770444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0897b396-ecde-4137-89e9-db69b7897515 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:36.771704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0897b396-ecde-4137-89e9-db69b7897515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:36.779586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 751/1483] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:36.817888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e4bb-b4a9-4e11-83f6-ad04f87b75ec None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:36.828559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e4bb-b4a9-4e11-83f6-ad04f87b75ec None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:36.832042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4f0e3-3f9e-4535-8fbc-c866e1d82d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:36.832680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f0e3-3f9e-4535-8fbc-c866e1d82d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:36.832973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f0e3-3f9e-4535-8fbc-c866e1d82d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:36.833255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f0e3-3f9e-4535-8fbc-c866e1d82d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:36.842176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4f0e3-3f9e-4535-8fbc-c866e1d82d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:36.843639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4f0e3-3f9e-4535-8fbc-c866e1d82d29 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:36.852862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 733/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:39.005548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2ad3f-da21-4d97-87f1-c5e137ab7622 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:39.006107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2ad3f-da21-4d97-87f1-c5e137ab7622 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:39.006107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2ad3f-da21-4d97-87f1-c5e137ab7622 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:39.007533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 752/1485] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:39.012777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c76a2-27bc-4f88-afb7-a24b5cd2bca8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:39.013211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c76a2-27bc-4f88-afb7-a24b5cd2bca8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:39.013376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c76a2-27bc-4f88-afb7-a24b5cd2bca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:39.031585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c64c76a2-27bc-4f88-afb7-a24b5cd2bca8 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:39.032785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c64c76a2-27bc-4f88-afb7-a24b5cd2bca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:39.040539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 734/1486] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:04:39.078941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2ad3f-da21-4d97-87f1-c5e137ab7622 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:39.090215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2ad3f-da21-4d97-87f1-c5e137ab7622 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:39.093709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a1f7b-0634-4c46-b862-3c070f104192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:39.094338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a1f7b-0634-4c46-b862-3c070f104192 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:39.094623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a1f7b-0634-4c46-b862-3c070f104192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:39.094896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a1f7b-0634-4c46-b862-3c070f104192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:39.103662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a1f7b-0634-4c46-b862-3c070f104192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:39.105125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a1f7b-0634-4c46-b862-3c070f104192 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:39.114314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 753/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:41.367716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db97b-79b0-442f-a505-cb128c9099ba None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:41.368282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db97b-79b0-442f-a505-cb128c9099ba None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:41.368282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049db97b-79b0-442f-a505-cb128c9099ba None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:41.370246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 735/1488] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:41.375662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9cb05-9220-4aa4-92a2-71c319453922 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:41.376093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9cb05-9220-4aa4-92a2-71c319453922 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:41.376250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9cb05-9220-4aa4-92a2-71c319453922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:41.393616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bfe9cb05-9220-4aa4-92a2-71c319453922 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:41.394798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bfe9cb05-9220-4aa4-92a2-71c319453922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:41.402552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 754/1489] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:41.443333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049db97b-79b0-442f-a505-cb128c9099ba None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:41.453978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049db97b-79b0-442f-a505-cb128c9099ba None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:41.457387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a5cd-fc47-46cd-a4ae-1c74927783c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:41.458014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a5cd-fc47-46cd-a4ae-1c74927783c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:41.458297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a5cd-fc47-46cd-a4ae-1c74927783c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:41.458575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a5cd-fc47-46cd-a4ae-1c74927783c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:41.468701 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a5cd-fc47-46cd-a4ae-1c74927783c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:41.470120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a5cd-fc47-46cd-a4ae-1c74927783c2 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:41.479896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 736/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:43.845139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa8f7-057b-44ea-a7d4-75230ab8fd04 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:43.845735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa8f7-057b-44ea-a7d4-75230ab8fd04 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:43.845735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fa8f7-057b-44ea-a7d4-75230ab8fd04 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:43.847083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 755/1491] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 11:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:04:43.852379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8e3ed-17ce-48b2-90b0-cd7229686b80 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:43.852812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8e3ed-17ce-48b2-90b0-cd7229686b80 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:43.852928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8e3ed-17ce-48b2-90b0-cd7229686b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:43.871735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-93c8e3ed-17ce-48b2-90b0-cd7229686b80 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:04:43.872953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-93c8e3ed-17ce-48b2-90b0-cd7229686b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:04:43.880743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 737/1492] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 11:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:04:43.920546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fa8f7-057b-44ea-a7d4-75230ab8fd04 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:04:43.931396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fa8f7-057b-44ea-a7d4-75230ab8fd04 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:04:43.934826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da06319-080a-4e84-8547-06c2ca35dca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:04:43.935512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da06319-080a-4e84-8547-06c2ca35dca1 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:04:43.935795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da06319-080a-4e84-8547-06c2ca35dca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:04:43.936071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da06319-080a-4e84-8547-06c2ca35dca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:04:43.944686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da06319-080a-4e84-8547-06c2ca35dca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:04:43.946213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da06319-080a-4e84-8547-06c2ca35dca1 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:04:43.955410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 756/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:04:53.865946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=33 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:04:53.951330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=33 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:23.464681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012be910-ea60-41c9-bb5e-8c674678b526 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:23.465292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012be910-ea60-41c9-bb5e-8c674678b526 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:23.465292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012be910-ea60-41c9-bb5e-8c674678b526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:23.488923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-012be910-ea60-41c9-bb5e-8c674678b526 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:23.490166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-012be910-ea60-41c9-bb5e-8c674678b526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:23.498435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 738/1494] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:06:23.508929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de0e90-94e2-4612-9c1d-d76a95270611 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:23.509488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de0e90-94e2-4612-9c1d-d76a95270611 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:23.509488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de0e90-94e2-4612-9c1d-d76a95270611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:23.528501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-85de0e90-94e2-4612-9c1d-d76a95270611 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:23.529776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-85de0e90-94e2-4612-9c1d-d76a95270611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:23.537795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 757/1495] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:23.542771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012be910-ea60-41c9-bb5e-8c674678b526 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:23.555390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012be910-ea60-41c9-bb5e-8c674678b526 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:23.560137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:23.560796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:23.561089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:23.561394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:23.562113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:06:23.562430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:23.563591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:23.563591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:06:23.563591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:06:23.564490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:23.577610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 739/1496] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:06:23.586588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de0e90-94e2-4612-9c1d-d76a95270611 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:23.595969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de0e90-94e2-4612-9c1d-d76a95270611 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:23.600648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:23.601299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:23.601582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:23.601859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:23.602545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:23.604355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:23.604355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jun 23 11:06:23.604355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: Jun 23 11:06:23.605551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:23.738764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 758/1497] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:06:23 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:06:23.743726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:23.753716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e807507-df62-4b97-a2ea-be7682d035f0 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:23.809830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:23.810497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:23.810786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:23.811067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:23.811797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:23.812809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:23.812809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:06:23.812809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:06:23.814034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:23.932739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 740/1498] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:06:23 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:06:23.936752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:23.947567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f9fc8-80d7-4815-8904-83f01f4d09b0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:23.952058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:23.952752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:23.953037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:23.953316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:23.954042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:23.955054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:23.955054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jun 23 11:06:23.955054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: Jun 23 11:06:23.955970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:23.968597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 759/1499] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:06:23.972451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:23.982351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a8fea-d34b-4636-8285-517da3737eb5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:23.986917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d935dcf-f20d-4003-9d1c-e95d408816cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:23.987610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d935dcf-f20d-4003-9d1c-e95d408816cb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:23.987901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d935dcf-f20d-4003-9d1c-e95d408816cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:23.988187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d935dcf-f20d-4003-9d1c-e95d408816cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:23.989464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d935dcf-f20d-4003-9d1c-e95d408816cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:23.990527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-4d935dcf-f20d-4003-9d1c-e95d408816cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:23.990527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:06:23.990527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:06:23.991432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d935dcf-f20d-4003-9d1c-e95d408816cb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:24.004450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 741/1500] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:06:24.008928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:24.018557 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364210a-3124-4324-ba83-ba380ad66974 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:24.023149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113796-96a3-45e1-bf80-6fa510291e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:24.024070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113796-96a3-45e1-bf80-6fa510291e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.024354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113796-96a3-45e1-bf80-6fa510291e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.024630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113796-96a3-45e1-bf80-6fa510291e46 admin admin] PATH_INFO: `/v3/projects/c8174f152ed9409e9740f9a4d0c37eca/users/bd51f25b842748c78c590fce66575865/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.039787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113796-96a3-45e1-bf80-6fa510291e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8174f152ed9409e9740f9a4d0c37eca, user_id=bd51f25b842748c78c590fce66575865, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:24.040810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-8c113796-96a3-45e1-bf80-6fa510291e46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:24.040810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jun 23 11:06:24.040810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: Jun 23 11:06:24.042738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c113796-96a3-45e1-bf80-6fa510291e46 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:24.108248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 760/1501] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:06:24 2022] PUT /identity/v3/projects/c8174f152ed9409e9740f9a4d0c37eca/users/bd51f25b842748c78c590fce66575865/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:06:24.120191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c8780-bc4f-46fd-a7f2-36d873ed4e09 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.120488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c8780-bc4f-46fd-a7f2-36d873ed4e09 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.120772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c8780-bc4f-46fd-a7f2-36d873ed4e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.146506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9e8c8780-bc4f-46fd-a7f2-36d873ed4e09 None None] MFA Rules not processed for user `bd51f25b842748c78c590fce66575865`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:24.147748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9e8c8780-bc4f-46fd-a7f2-36d873ed4e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:24.212743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 742/1502] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:24.226802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d99cb-a690-4198-adbe-634077e30ed8 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.227098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d99cb-a690-4198-adbe-634077e30ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.227408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d99cb-a690-4198-adbe-634077e30ed8 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.228794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 761/1503] 10.222.0.43 () {54 vars in 963 bytes} [Thu Jun 23 11:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:06:24.233786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006e32a-ce51-4949-bd16-7927de273810 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.234087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006e32a-ce51-4949-bd16-7927de273810 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.234375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006e32a-ce51-4949-bd16-7927de273810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.253431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c006e32a-ce51-4949-bd16-7927de273810 None None] MFA Rules not processed for user `2678deaa97fc4012aa0d71f9e505f5ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:24.254582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c006e32a-ce51-4949-bd16-7927de273810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:24.299447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:24.307735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 743/1504] 10.222.0.43 () {58 vars in 1060 bytes} [Thu Jun 23 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:24.315497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d99cb-a690-4198-adbe-634077e30ed8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:24.326023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d99cb-a690-4198-adbe-634077e30ed8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:24.329608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:24.330242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.330525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.330804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.339124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:24.340589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:24.350027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 762/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:06:24.392008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bfdff-8c57-4081-b4a4-28aa5e4f7b11 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.392304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bfdff-8c57-4081-b4a4-28aa5e4f7b11 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.392588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bfdff-8c57-4081-b4a4-28aa5e4f7b11 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.393895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 744/1506] 10.222.0.43 () {52 vars in 829 bytes} [Thu Jun 23 11:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:06:24.398451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:24.461475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce866c-f140-49e4-b49c-7d63c713d441 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:24.465990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:24.466859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.467146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.467456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] PATH_INFO: `/v3/users/bd51f25b842748c78c590fce66575865` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.468639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd51f25b842748c78c590fce66575865)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:24.469661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:24.469661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jun 23 11:06:24.469661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: Jun 23 11:06:24.470872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:24.727327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd51f25b842748c78c590fce66575865'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:24.740074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd51f25b842748c78c590fce66575865'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:24.740877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd51f25b842748c78c590fce66575865'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:24.741543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd51f25b842748c78c590fce66575865'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:24.760285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd51f25b842748c78c590fce66575865'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:24.764233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:24.862936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 763/1507] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:06:24 2022] DELETE /identity/v3/users/bd51f25b842748c78c590fce66575865 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:06:24.867656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:24.868145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:24.868252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:24.886076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:06:24.887492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:24.947674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 745/1508] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:06:24.957784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:24.959776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:25.018180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3e7f5-a4b3-4e5b-9df0-5c5fb7214e7d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:25.022689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a60ad4-4d89-40f9-9565-85379c4cf42d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:25.023369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a60ad4-4d89-40f9-9565-85379c4cf42d service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:25.023658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a60ad4-4d89-40f9-9565-85379c4cf42d service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:25.023938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a60ad4-4d89-40f9-9565-85379c4cf42d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:25.033278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a60ad4-4d89-40f9-9565-85379c4cf42d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:25.034752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a60ad4-4d89-40f9-9565-85379c4cf42d service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:25.044739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 764/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:06:25.567429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:06:25.568844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:06:25.580986 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5d811-06fd-4290-bf30-07449b630697 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:06:25.586144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:06:25.589044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:06:25.589044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:06:25.589044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] PATH_INFO: `/v3/projects/c8174f152ed9409e9740f9a4d0c37eca` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:06:25.589044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8174f152ed9409e9740f9a4d0c37eca)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:06:25.589812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:06:25.589812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:06:25.589812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:06:25.591605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:06:25.725123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8174f152ed9409e9740f9a4d0c37eca', 'request_id': 'req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:25.731059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8174f152ed9409e9740f9a4d0c37eca', 'request_id': 'req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:25.745328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8174f152ed9409e9740f9a4d0c37eca', 'request_id': 'req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:25.839693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] The token cache is being invalidate because project c8174f152ed9409e9740f9a4d0c37eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:06:25.839987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:25.840716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-22d9824b-bd93-47fd-92ba-66d2d7fe27bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:06:25.842507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 746/1510] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:06:25 2022] DELETE /identity/v3/projects/c8174f152ed9409e9740f9a4d0c37eca => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:06:35.040462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:06:35.836454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:05.190646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c565c4f-f202-44b6-92cc-f648bf0181ce None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:05.191231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c565c4f-f202-44b6-92cc-f648bf0181ce None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:05.191231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c565c4f-f202-44b6-92cc-f648bf0181ce None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:05.192533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:05.197648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8dba9-6b2c-4921-9eea-79e7229b3a49 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:05.198072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8dba9-6b2c-4921-9eea-79e7229b3a49 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:05.198215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8dba9-6b2c-4921-9eea-79e7229b3a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:05.217010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-45a8dba9-6b2c-4921-9eea-79e7229b3a49 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:05.218279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-45a8dba9-6b2c-4921-9eea-79e7229b3a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:05.276987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:05.432092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c565c4f-f202-44b6-92cc-f648bf0181ce None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:05.433807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c565c4f-f202-44b6-92cc-f648bf0181ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:05.509434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c565c4f-f202-44b6-92cc-f648bf0181ce None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:05.514710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6154a-87cc-49a5-a41f-e0b2f4d53f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:05.515374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6154a-87cc-49a5-a41f-e0b2f4d53f28 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:05.515686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6154a-87cc-49a5-a41f-e0b2f4d53f28 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:05.515941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6154a-87cc-49a5-a41f-e0b2f4d53f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:05.524560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6154a-87cc-49a5-a41f-e0b2f4d53f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:05.526152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6154a-87cc-49a5-a41f-e0b2f4d53f28 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:05.535812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 766/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:06.358657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443702-3ce9-40c5-8676-209373029f4d None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:06.359234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443702-3ce9-40c5-8676-209373029f4d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:06.359234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49443702-3ce9-40c5-8676-209373029f4d None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:06.360666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:06.366376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11119-f3f6-4b17-a975-2d592b53e551 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:06.367157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11119-f3f6-4b17-a975-2d592b53e551 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:06.367479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd11119-f3f6-4b17-a975-2d592b53e551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:06.385255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2cd11119-f3f6-4b17-a975-2d592b53e551 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:06.386459 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2cd11119-f3f6-4b17-a975-2d592b53e551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:06.394471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:06.548888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443702-3ce9-40c5-8676-209373029f4d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:06.563927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49443702-3ce9-40c5-8676-209373029f4d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:06.569361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e53cf-1198-4521-bfcd-cb0a8ecf56fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:06.569995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e53cf-1198-4521-bfcd-cb0a8ecf56fb service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:06.570284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e53cf-1198-4521-bfcd-cb0a8ecf56fb service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:06.570565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e53cf-1198-4521-bfcd-cb0a8ecf56fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:06.578846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e53cf-1198-4521-bfcd-cb0a8ecf56fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:06.580494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e53cf-1198-4521-bfcd-cb0a8ecf56fb service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:06.590460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 749/1516] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:07.464522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de24c8-c7d4-44c9-94f2-be0c5079da91 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.465091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de24c8-c7d4-44c9-94f2-be0c5079da91 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.465091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de24c8-c7d4-44c9-94f2-be0c5079da91 None None] PATH_INFO: `/v3` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.466380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 768/1517] 10.222.0.18 () {54 vars in 925 bytes} [Thu Jun 23 11:08:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:08:07.472422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936974d-fc6e-43b2-917a-647e01c9b3c8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:07.472818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936974d-fc6e-43b2-917a-647e01c9b3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:07.473203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936974d-fc6e-43b2-917a-647e01c9b3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:07.491913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c936974d-fc6e-43b2-917a-647e01c9b3c8 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:07.493111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c936974d-fc6e-43b2-917a-647e01c9b3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:07.555506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 750/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jun 23 11:08:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:11.538414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39cfe5-9f75-43eb-a663-82d484b74135 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:11.538938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39cfe5-9f75-43eb-a663-82d484b74135 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:11.538938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39cfe5-9f75-43eb-a663-82d484b74135 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:11.540275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 769/1519] 10.222.0.43 () {54 vars in 960 bytes} [Thu Jun 23 11:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:11.545489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae383d-363f-4cb4-a486-104e0a2d90d3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:11.545885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae383d-363f-4cb4-a486-104e0a2d90d3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:11.546051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae383d-363f-4cb4-a486-104e0a2d90d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:11.563737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-35ae383d-363f-4cb4-a486-104e0a2d90d3 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:11.564958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-35ae383d-363f-4cb4-a486-104e0a2d90d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:11.574027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 751/1520] 10.222.0.43 () {58 vars in 1057 bytes} [Thu Jun 23 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:11.581436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39cfe5-9f75-43eb-a663-82d484b74135 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:11.595121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39cfe5-9f75-43eb-a663-82d484b74135 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:11.602523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabfc35-0cc1-40ec-b1d0-e474aa37f791 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:11.602523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabfc35-0cc1-40ec-b1d0-e474aa37f791 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:11.602523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabfc35-0cc1-40ec-b1d0-e474aa37f791 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:11.602523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabfc35-0cc1-40ec-b1d0-e474aa37f791 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:11.609407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabfc35-0cc1-40ec-b1d0-e474aa37f791 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:11.610973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabfc35-0cc1-40ec-b1d0-e474aa37f791 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:11.620570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 770/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:17.833621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391653c-eb9f-47ad-8aa8-847b2466be25 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:17.834192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391653c-eb9f-47ad-8aa8-847b2466be25 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:17.834192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391653c-eb9f-47ad-8aa8-847b2466be25 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:17.835616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:17.840587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686e6ab-db7a-4d36-b2ea-860e13eab414 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:17.841028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686e6ab-db7a-4d36-b2ea-860e13eab414 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:17.841157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686e6ab-db7a-4d36-b2ea-860e13eab414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:17.858845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2686e6ab-db7a-4d36-b2ea-860e13eab414 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:17.860068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2686e6ab-db7a-4d36-b2ea-860e13eab414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:17.868408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:18.024212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391653c-eb9f-47ad-8aa8-847b2466be25 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:18.038520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391653c-eb9f-47ad-8aa8-847b2466be25 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:18.045014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e321edf-0a55-4f2f-b238-115d9878aab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:18.045649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e321edf-0a55-4f2f-b238-115d9878aab0 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:18.045934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e321edf-0a55-4f2f-b238-115d9878aab0 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:18.046211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e321edf-0a55-4f2f-b238-115d9878aab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:18.055390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e321edf-0a55-4f2f-b238-115d9878aab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:18.057004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e321edf-0a55-4f2f-b238-115d9878aab0 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:18.066363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 753/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:18.882862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334d797-13bf-484f-a716-59cf4f0f5d5e None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:18.883443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334d797-13bf-484f-a716-59cf4f0f5d5e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:18.883443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334d797-13bf-484f-a716-59cf4f0f5d5e None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:18.884873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:18.889585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9cbab-643f-402e-9a3c-8d9280246270 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:18.890027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9cbab-643f-402e-9a3c-8d9280246270 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:18.890134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9cbab-643f-402e-9a3c-8d9280246270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:18.908860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-bcf9cbab-643f-402e-9a3c-8d9280246270 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:18.910075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bcf9cbab-643f-402e-9a3c-8d9280246270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:18.918850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:19.080951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334d797-13bf-484f-a716-59cf4f0f5d5e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:19.096295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334d797-13bf-484f-a716-59cf4f0f5d5e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:19.101922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef23543-6c6b-4788-8a72-5b9449c1d033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:19.102566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef23543-6c6b-4788-8a72-5b9449c1d033 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:19.102860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef23543-6c6b-4788-8a72-5b9449c1d033 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:19.103145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef23543-6c6b-4788-8a72-5b9449c1d033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:19.111274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef23543-6c6b-4788-8a72-5b9449c1d033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:19.112897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef23543-6c6b-4788-8a72-5b9449c1d033 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:19.122768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 773/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:23.169392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca91516-bab9-4cd8-95c4-0c4e7674193a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:23.169979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca91516-bab9-4cd8-95c4-0c4e7674193a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:23.169979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca91516-bab9-4cd8-95c4-0c4e7674193a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:23.171317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:23.178176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd34ec7-68d3-4476-8164-a910d0d85f99 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:23.178788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd34ec7-68d3-4476-8164-a910d0d85f99 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:23.178788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd34ec7-68d3-4476-8164-a910d0d85f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:23.197569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4fd34ec7-68d3-4476-8164-a910d0d85f99 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:23.198755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4fd34ec7-68d3-4476-8164-a910d0d85f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:23.206499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:23.243468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca91516-bab9-4cd8-95c4-0c4e7674193a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:23.245159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ca91516-bab9-4cd8-95c4-0c4e7674193a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:23.305691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca91516-bab9-4cd8-95c4-0c4e7674193a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:23.309223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e903259-e8d1-4c5e-887e-84fe32dd5088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:23.309874 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e903259-e8d1-4c5e-887e-84fe32dd5088 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:23.310163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e903259-e8d1-4c5e-887e-84fe32dd5088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:23.310444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e903259-e8d1-4c5e-887e-84fe32dd5088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:23.319310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e903259-e8d1-4c5e-887e-84fe32dd5088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:23.320916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e903259-e8d1-4c5e-887e-84fe32dd5088 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:23.325825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:23.330491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 756/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:25.348299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6309c-8f0d-4a0e-9b3f-50ecb7a959d7 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:25.348871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6309c-8f0d-4a0e-9b3f-50ecb7a959d7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:25.348999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6309c-8f0d-4a0e-9b3f-50ecb7a959d7 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:25.350803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:25.356104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff05f-9160-4f07-b871-8200765e1e3c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:25.356564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff05f-9160-4f07-b871-8200765e1e3c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:25.356672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ff05f-9160-4f07-b871-8200765e1e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:25.494983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2d6ff05f-9160-4f07-b871-8200765e1e3c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:25.496243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2d6ff05f-9160-4f07-b871-8200765e1e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:25.504054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:25.540222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6309c-8f0d-4a0e-9b3f-50ecb7a959d7 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:25.551408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6309c-8f0d-4a0e-9b3f-50ecb7a959d7 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:25.555718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7272ae-b4d4-4f77-ab27-7c523f4d4612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:25.556380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7272ae-b4d4-4f77-ab27-7c523f4d4612 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:25.556753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7272ae-b4d4-4f77-ab27-7c523f4d4612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:25.557035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7272ae-b4d4-4f77-ab27-7c523f4d4612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:25.566047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7272ae-b4d4-4f77-ab27-7c523f4d4612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:25.567508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7272ae-b4d4-4f77-ab27-7c523f4d4612 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:25.577698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 776/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:29.283313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a6c07-2f7d-483f-997c-da809419a246 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:29.283850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a6c07-2f7d-483f-997c-da809419a246 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:29.283850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a6c07-2f7d-483f-997c-da809419a246 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:29.285205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:29.290959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba07b-2560-4747-a6c9-a00688b10db6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:29.291448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba07b-2560-4747-a6c9-a00688b10db6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:29.291553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba07b-2560-4747-a6c9-a00688b10db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:29.309592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bc8ba07b-2560-4747-a6c9-a00688b10db6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:29.310781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bc8ba07b-2560-4747-a6c9-a00688b10db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:29.318548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:08:29.636169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a6c07-2f7d-483f-997c-da809419a246 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:29.637822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fd1a6c07-2f7d-483f-997c-da809419a246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:29.694710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a6c07-2f7d-483f-997c-da809419a246 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:29.698786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb26bd-7776-404a-a814-d89b23987f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:29.699457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb26bd-7776-404a-a814-d89b23987f94 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:29.699745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb26bd-7776-404a-a814-d89b23987f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:29.700020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb26bd-7776-404a-a814-d89b23987f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:29.708838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb26bd-7776-404a-a814-d89b23987f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:29.710257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb26bd-7776-404a-a814-d89b23987f94 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:29.720327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 759/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:31.759881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4cb21-0776-4ac7-96e4-092e09d376a1 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:31.760464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4cb21-0776-4ac7-96e4-092e09d376a1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:31.760464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4cb21-0776-4ac7-96e4-092e09d376a1 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:31.761829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:08:31.767307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523e56e-a528-400b-bc18-14c922e01e39 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:31.767788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523e56e-a528-400b-bc18-14c922e01e39 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:31.767884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523e56e-a528-400b-bc18-14c922e01e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:31.786207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3523e56e-a528-400b-bc18-14c922e01e39 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:08:31.787430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3523e56e-a528-400b-bc18-14c922e01e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:31.796565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:08:32.106034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4cb21-0776-4ac7-96e4-092e09d376a1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:08:32.107728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-93f4cb21-0776-4ac7-96e4-092e09d376a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:08:32.121267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4cb21-0776-4ac7-96e4-092e09d376a1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:08:32.124762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d789119-4990-4398-8dab-06eabbbd568b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:08:32.125402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d789119-4990-4398-8dab-06eabbbd568b service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:08:32.125690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d789119-4990-4398-8dab-06eabbbd568b service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:08:32.125967 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d789119-4990-4398-8dab-06eabbbd568b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:08:32.135673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d789119-4990-4398-8dab-06eabbbd568b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:08:32.137113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d789119-4990-4398-8dab-06eabbbd568b service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:08:32.146599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 779/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:08:41.781293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=32 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:08:42.142347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=70 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:09:59.615340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ded591-b7a9-4238-8430-be66fa402f7a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.617850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ded591-b7a9-4238-8430-be66fa402f7a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.617850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ded591-b7a9-4238-8430-be66fa402f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.643735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d5ded591-b7a9-4238-8430-be66fa402f7a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:59.644972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d5ded591-b7a9-4238-8430-be66fa402f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:59.659279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 761/1540] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:09:59.669493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f70e0-0ff3-4755-8ca4-36c47b62b15e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.670512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f70e0-0ff3-4755-8ca4-36c47b62b15e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.671019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f70e0-0ff3-4755-8ca4-36c47b62b15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.703427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c81f70e0-0ff3-4755-8ca4-36c47b62b15e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:09:59.705038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c81f70e0-0ff3-4755-8ca4-36c47b62b15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:09:59.725026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 780/1541] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:09:59.730521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ded591-b7a9-4238-8430-be66fa402f7a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.745289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ded591-b7a9-4238-8430-be66fa402f7a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.755494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.756171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.756456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.756730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.758213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:09:59.758414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.760208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.778719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 762/1542] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:09:59.786637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f70e0-0ff3-4755-8ca4-36c47b62b15e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.803200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f70e0-0ff3-4755-8ca4-36c47b62b15e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:09:59.809416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef1e79-97f5-435d-9315-527cf2907975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:09:59.810386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef1e79-97f5-435d-9315-527cf2907975 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:09:59.810940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef1e79-97f5-435d-9315-527cf2907975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:09:59.811498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef1e79-97f5-435d-9315-527cf2907975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:09:59.812442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef1e79-97f5-435d-9315-527cf2907975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:09:59.814683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef1e79-97f5-435d-9315-527cf2907975 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:09:59.937491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 781/1543] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:09:59.942535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:09:59.957139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378cc1-a831-41e8-96ce-d05dee395218 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.035828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39275847-8d97-48df-b5fe-8b3a8e6c4b70 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.036525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39275847-8d97-48df-b5fe-8b3a8e6c4b70 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.037286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39275847-8d97-48df-b5fe-8b3a8e6c4b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.047912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de9fe0-ba19-4f48-890f-c2fa972f156a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.048580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9fe0-ba19-4f48-890f-c2fa972f156a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.048865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9fe0-ba19-4f48-890f-c2fa972f156a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.049142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de9fe0-ba19-4f48-890f-c2fa972f156a admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.049838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de9fe0-ba19-4f48-890f-c2fa972f156a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.051795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de9fe0-ba19-4f48-890f-c2fa972f156a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.079672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-39275847-8d97-48df-b5fe-8b3a8e6c4b70 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.083869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-39275847-8d97-48df-b5fe-8b3a8e6c4b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.095450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 782/1544] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.097223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cb077-5741-434f-bc18-733fb587cd01 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.097736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cb077-5741-434f-bc18-733fb587cd01 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.098248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cb077-5741-434f-bc18-733fb587cd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.132314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-421cb077-5741-434f-bc18-733fb587cd01 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.137000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-421cb077-5741-434f-bc18-733fb587cd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.147268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 783/1545] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.149657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b34de-aee3-4f9e-8b71-44a0e46c5104 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.150198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b34de-aee3-4f9e-8b71-44a0e46c5104 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.150700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b34de-aee3-4f9e-8b71-44a0e46c5104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.170753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 763/1546] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:09:59 2022] POST /identity/v3/users => generated 282 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:00.170975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-247b34de-aee3-4f9e-8b71-44a0e46c5104 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.174092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30045-b172-4a78-a393-0bb90e985968 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.174092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30045-b172-4a78-a393-0bb90e985968 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.174092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e30045-b172-4a78-a393-0bb90e985968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.181907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-247b34de-aee3-4f9e-8b71-44a0e46c5104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.195986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 784/1547] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.202073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec8a8-d12a-4ba3-979b-8ee739192157 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.202734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec8a8-d12a-4ba3-979b-8ee739192157 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.203259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec8a8-d12a-4ba3-979b-8ee739192157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.215776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-34e30045-b172-4a78-a393-0bb90e985968 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.217043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-34e30045-b172-4a78-a393-0bb90e985968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.232601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 764/1548] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.233405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-688ec8a8-d12a-4ba3-979b-8ee739192157 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.234556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e30045-b172-4a78-a393-0bb90e985968 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.234691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-688ec8a8-d12a-4ba3-979b-8ee739192157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.247786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 785/1549] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.250307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e30045-b172-4a78-a393-0bb90e985968 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.252417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f634b-4254-4a71-85b8-7213706c23f5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.252713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f634b-4254-4a71-85b8-7213706c23f5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.252999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f634b-4254-4a71-85b8-7213706c23f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.256080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.257151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.258131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.258572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.259492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.261268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.271996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 765/1550] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:00.273112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.275797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d43f634b-4254-4a71-85b8-7213706c23f5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.277011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d43f634b-4254-4a71-85b8-7213706c23f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.284213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcd212-d903-42cc-9e4d-c07f0d3de9e4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.284971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 786/1551] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.286341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a153ffa-278f-4308-9518-dd4d174ceeb1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.286634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a153ffa-278f-4308-9518-dd4d174ceeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.287163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a153ffa-278f-4308-9518-dd4d174ceeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.289960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d18dca-81d8-4aa3-8e46-b1bb8c84a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.290586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d18dca-81d8-4aa3-8e46-b1bb8c84a373 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.290871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d18dca-81d8-4aa3-8e46-b1bb8c84a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.291149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d18dca-81d8-4aa3-8e46-b1bb8c84a373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.291897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d18dca-81d8-4aa3-8e46-b1bb8c84a373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:00.292202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d18dca-81d8-4aa3-8e46-b1bb8c84a373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.293786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d18dca-81d8-4aa3-8e46-b1bb8c84a373 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.306645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8a153ffa-278f-4308-9518-dd4d174ceeb1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.307417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 766/1552] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:00.308726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3784641-8dbc-4747-9467-e0acf02984c1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.309015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3784641-8dbc-4747-9467-e0acf02984c1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.309291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3784641-8dbc-4747-9467-e0acf02984c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.312093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8a153ffa-278f-4308-9518-dd4d174ceeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.323579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 787/1553] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.325393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5a52a-a160-4b82-a15c-76482a6f417c None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.326045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5a52a-a160-4b82-a15c-76482a6f417c None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.326561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5a52a-a160-4b82-a15c-76482a6f417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.331492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f3784641-8dbc-4747-9467-e0acf02984c1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.332886 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f3784641-8dbc-4747-9467-e0acf02984c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.344339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 767/1554] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.345080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3784641-8dbc-4747-9467-e0acf02984c1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.346573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bad5a52a-a160-4b82-a15c-76482a6f417c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.347895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bad5a52a-a160-4b82-a15c-76482a6f417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.356920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3784641-8dbc-4747-9467-e0acf02984c1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.359022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 788/1555] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.359903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5a52a-a160-4b82-a15c-76482a6f417c None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.362020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8d62e-b69e-411e-8d2b-4ceb79fb81c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.362905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d62e-b69e-411e-8d2b-4ceb79fb81c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.363197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d62e-b69e-411e-8d2b-4ceb79fb81c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.363488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d62e-b69e-411e-8d2b-4ceb79fb81c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.364222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8d62e-b69e-411e-8d2b-4ceb79fb81c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:00.364543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8d62e-b69e-411e-8d2b-4ceb79fb81c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.366131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8d62e-b69e-411e-8d2b-4ceb79fb81c7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.369558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5a52a-a160-4b82-a15c-76482a6f417c None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.374620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.375322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.375613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.376027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.376651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.378044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 768/1556] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:00.378273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.379405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68996-bcc7-453e-b63a-89d753a7a489 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.379731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68996-bcc7-453e-b63a-89d753a7a489 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.379985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68996-bcc7-453e-b63a-89d753a7a489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.391307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 789/1557] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:00.391307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.398673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c3b68996-bcc7-453e-b63a-89d753a7a489 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.400056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c3b68996-bcc7-453e-b63a-89d753a7a489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.401478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae42f0-2066-43b0-9b24-46377f708890 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.406055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.406707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.407001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.407311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.407852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 769/1558] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.409204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b6687-db9f-4c63-84eb-9b4cfc8a5d91 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.409498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b6687-db9f-4c63-84eb-9b4cfc8a5d91 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.409782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b6687-db9f-4c63-84eb-9b4cfc8a5d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.411436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.413765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.428671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1f8b6687-db9f-4c63-84eb-9b4cfc8a5d91 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.429967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1f8b6687-db9f-4c63-84eb-9b4cfc8a5d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.437768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 770/1559] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.440290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020543-9b2e-4983-a456-c5b03a510b6b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.440581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020543-9b2e-4983-a456-c5b03a510b6b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.440861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020543-9b2e-4983-a456-c5b03a510b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.458545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b9020543-9b2e-4983-a456-c5b03a510b6b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.459730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b9020543-9b2e-4983-a456-c5b03a510b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.468384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 771/1560] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.468982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0828cca-2c66-41c7-8073-d006afa851c5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.469270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0828cca-2c66-41c7-8073-d006afa851c5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.469554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0828cca-2c66-41c7-8073-d006afa851c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.487530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d0828cca-2c66-41c7-8073-d006afa851c5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:00.488717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d0828cca-2c66-41c7-8073-d006afa851c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:00.496562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 772/1561] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:00.500404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0828cca-2c66-41c7-8073-d006afa851c5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.506490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0828cca-2c66-41c7-8073-d006afa851c5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.511033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.511686 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.511974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.512254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.512971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:00.513279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.514917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.525964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 790/1562] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:00.525964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.531415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 773/1563] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:00.531415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.542908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3d21d-21f4-4896-914a-976a5a4bc0b2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.582811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:00.589307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62d46c-e312-487e-92db-59577df79304 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.593902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.594563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.594848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.595126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.595847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.597757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.602509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.603480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.603754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.604035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] PATH_INFO: `/v3/projects/24b0df2cbf744712b463758fcf18a5f5/users/b0b7dc063abb4e4fabbed015d9cf73df/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.617918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24b0df2cbf744712b463758fcf18a5f5, user_id=b0b7dc063abb4e4fabbed015d9cf73df, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.622914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:10:00.622914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:10:00.622914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:10:00.622914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.708170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 791/1564] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:00.709112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.741168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 774/1565] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:00 2022] PUT /identity/v3/projects/24b0df2cbf744712b463758fcf18a5f5/users/b0b7dc063abb4e4fabbed015d9cf73df/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:00.742177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.752958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:00.774705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351aa95d-13b3-44de-bb5d-947d14b69e6e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.903524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.904260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.904559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.905412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.905598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:00.905909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.906933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:10:00.906933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jun 23 11:10:00.906933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: Jun 23 11:10:00.907872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaddfb4-b190-4e52-b1da-362b059a1c24 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.908806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbaf47-43ec-4e4b-803c-990c29017e49 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.917941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.918625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.918914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.919266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.920014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:00.920346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.920643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 792/1566] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:00.922099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3e0a4-c660-4d0f-8972-ffbaa8c11c6e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.922219 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.922305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3e0a4-c660-4d0f-8972-ffbaa8c11c6e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.922589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3e0a4-c660-4d0f-8972-ffbaa8c11c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.938946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 775/1567] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:00.939948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.950418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00d129-552c-40db-a406-065c4a19b6cb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.954897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.955560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.955848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.956183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.956896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:00.957205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.958875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:00.971566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 776/1568] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:00.972334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:00.981714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aac519-5ab5-465a-9930-4a5ed3f3247b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:00.986284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:00.986938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:00.987322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:00.987612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:00.988339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:00.990170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.007028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-84c3e0a4-c660-4d0f-8972-ffbaa8c11c6e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:01.012655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-84c3e0a4-c660-4d0f-8972-ffbaa8c11c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:01.017121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 793/1569] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:01.017861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3e0a4-c660-4d0f-8972-ffbaa8c11c6e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.027988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3e0a4-c660-4d0f-8972-ffbaa8c11c6e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.032460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996637-6086-496a-afab-79c3ebf436b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.033098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996637-6086-496a-afab-79c3ebf436b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.033381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996637-6086-496a-afab-79c3ebf436b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.033705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996637-6086-496a-afab-79c3ebf436b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.034384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996637-6086-496a-afab-79c3ebf436b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.036313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996637-6086-496a-afab-79c3ebf436b9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.103782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 777/1570] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:00 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.104751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.114092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6f28d-ea0f-442d-adc3-a6efec5813c9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.119912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.120584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.120868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.121143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.121821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.123754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.139787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 794/1571] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.143663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e480a6-fca0-4730-95ec-1f78d450940a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.143663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e480a6-fca0-4730-95ec-1f78d450940a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.143663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e480a6-fca0-4730-95ec-1f78d450940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.167804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-07e480a6-fca0-4730-95ec-1f78d450940a None None] MFA Rules not processed for user `b0b7dc063abb4e4fabbed015d9cf73df`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:01.170427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-07e480a6-fca0-4730-95ec-1f78d450940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:01.223547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 778/1572] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.224286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.234000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28052414-2c11-419c-af63-8f58c9308b50 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.236085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 795/1573] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:01.236947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e480a6-fca0-4730-95ec-1f78d450940a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.280421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:01.295777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.296532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.297115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.297115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.297641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.298662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:10:01.298662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:10:01.298662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:10:01.299873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.300807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e480a6-fca0-4730-95ec-1f78d450940a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.305930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.306595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.306916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.307223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.307946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.309904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.313138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:01.442268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 779/1574] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/projects => generated 363 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.446171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 796/1575] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.446329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.446426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.506560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef4e82-d215-4ed2-ad67-5d25509aa631 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.510512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4a04-6bbc-4d17-89b4-307af72e08e8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.511924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.512586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.512874 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.513154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.513843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.515202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.515793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.515899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.516215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.516501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.517233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:01.517540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.519132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.531096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 797/1576] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:01.531989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.542539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fb510-2771-4ea7-b11f-c25bc5263094 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.548481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.548481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.548481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.548481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.549011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.550564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.561105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 798/1577] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:01.561849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.570926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef80e8-d6f1-4b56-a3a0-723f10cb9b11 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.577581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.577581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.577581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.577581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.578025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.578492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:10:01.578492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jun 23 11:10:01.578492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: Jun 23 11:10:01.579716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.627105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 780/1578] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.628583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.691054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072ae5f-ee30-4dfb-a84f-43a50de2496a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.695802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.696556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.696854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.697134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.698087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.699710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.709506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 781/1579] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:01.710347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.713458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 799/1580] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.713458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.722213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab801e-c7f5-4adf-ac59-60e9618a6843 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.722359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820e752-42b0-417f-ac2b-011b78489600 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.727478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.727634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.728131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.728236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.728420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.728528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.728703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.728817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.729391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.729490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.731304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.731421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.809879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 800/1581] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.810674 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.820209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fa51e-bed7-439f-a3e3-67fa3426c782 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.824712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.825371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.825653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.825931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.826596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.828515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.842436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 782/1582] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.842436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.891455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:01.902502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57482c-bd23-47f2-93c8-984e236c4235 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.907242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.907951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.908212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.908487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.909171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.911067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.935014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 801/1583] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:01.935869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.945456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb951b0d-4933-4a6c-8bf7-f5f460ef0a03 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.949916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.950583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.950867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.951142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.951885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.953468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:01.964255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 802/1584] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:01.964999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:01.974232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef9933-c4f8-45e4-9d04-29c2eda01e42 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:01.979383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:01.980001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:01.980285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:01.980564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:01.981228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:01.983112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.001932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 783/1585] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:02.002798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.064184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7128e4d-1d61-4f07-8db5-c68f0d485470 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.069233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.070681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.070983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.071305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.072043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.073647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.083715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 784/1586] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.088990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.092532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 803/1587] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:01 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:02.092532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.098126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26e937-d82f-486e-8206-205f5d5266d8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.101678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ff1f2-f9fd-4261-bdec-d89f535edc13 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.103029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.103728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.104016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.104514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.105153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.106399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.106755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.107055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.107441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.107720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.108472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.110051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.117567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 785/1588] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.118437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.121248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 804/1589] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.122068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.128771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06cd0b-c062-4ba7-af01-a411df404ddc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.132965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0739e9-7880-48a9-8657-8a36dfc5662a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.134185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.134850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.135141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.135456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.136218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.137689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.138077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.138339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.138623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.138901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.139664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.141273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.152714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 805/1590] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.157671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.159571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:02.163764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fafbff-d55f-4068-9f33-acd794fc6c2b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.168538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.169400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.169689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.169971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] PATH_INFO: `/v3/projects/515a0335f2e04273b6efd97405f6c7b8/users/5e7ec74968f149f78334f707f7ff501a/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.186019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] RBAC: Authorizing `identity:create_grant(project_id=515a0335f2e04273b6efd97405f6c7b8, user_id=5e7ec74968f149f78334f707f7ff501a, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.188737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.216174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 786/1591] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:02.217128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.226800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff2ccc-38fc-4ef2-82e9-3d2af0b4abcb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.231357 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.231992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.232294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.232553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.233252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.235137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.295448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 806/1592] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/515a0335f2e04273b6efd97405f6c7b8/users/5e7ec74968f149f78334f707f7ff501a/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.296397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.318905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 787/1593] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:02.319713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.330752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf335b-b08a-42c0-ac6f-469f8f1afc0a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.359747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebbb7c-9fb7-4469-b6e6-455d4793785d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.364484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.365379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.365666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.365949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] PATH_INFO: `/v3/projects/03469431b9b64579b92f8714a35397c7/users/95f0f1dea22d4870b8b117c3be6dcd5a/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.382132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03469431b9b64579b92f8714a35397c7, user_id=95f0f1dea22d4870b8b117c3be6dcd5a, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.385126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.387625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.388304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.388588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.388870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.389602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.391210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.403536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 788/1594] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.403536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.411436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691370b-9bf5-40df-bc9b-7806b010e352 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.416760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.417379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.417665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.417945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.418647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.420228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.430395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 789/1595] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.431317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.440448 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395515-5189-4b97-8b17-4b5eed44f3da admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.445208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.445950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.446239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.446516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.447228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.448757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.455308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 807/1596] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/03469431b9b64579b92f8714a35397c7/users/95f0f1dea22d4870b8b117c3be6dcd5a/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.456204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.462719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 790/1597] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.462719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.468272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:02.473550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fc541-54b8-4b00-8a3a-433f0b5f3afa admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.519798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362eb879-ce99-4005-9ba3-057512132b4b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.524832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c1039-d44d-45b8-933b-fdc9bc857fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.525506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c1039-d44d-45b8-933b-fdc9bc857fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.525797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c1039-d44d-45b8-933b-fdc9bc857fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.526094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c1039-d44d-45b8-933b-fdc9bc857fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.526821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c1039-d44d-45b8-933b-fdc9bc857fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.528495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c1039-d44d-45b8-933b-fdc9bc857fe9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.533087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.533766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.534059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.534343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.535072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.536708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.540275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 808/1598] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.541647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01212c-1d4e-4867-a2a1-b4f0251ba505 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.541947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01212c-1d4e-4867-a2a1-b4f0251ba505 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.542226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01212c-1d4e-4867-a2a1-b4f0251ba505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.550849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 791/1599] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.550849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.558932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4433971-3359-493d-b616-5f6253a2661d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.564607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.570215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.570215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.570215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.570215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.570215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.573662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-eb01212c-1d4e-4867-a2a1-b4f0251ba505 None None] MFA Rules not processed for user `5e7ec74968f149f78334f707f7ff501a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:02.574869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-eb01212c-1d4e-4867-a2a1-b4f0251ba505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:02.580198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 792/1600] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.581037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.591369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870535b-4321-401d-9a4d-5cbacb5956e9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.596030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.596661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.596957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.597247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.597982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.599628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.611226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 793/1601] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:02.617358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.622740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb035e4-80d1-4fe2-b8ec-7eeb7027fd2f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.623333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:02.627380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19f493-8082-486b-80af-2c0d29e57c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.628461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f493-8082-486b-80af-2c0d29e57c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.628750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f493-8082-486b-80af-2c0d29e57c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.629035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f493-8082-486b-80af-2c0d29e57c0c admin admin] PATH_INFO: `/v3/projects/50f0e8f52e794a398b6b5db5634cf60a/users/3dd72ff391dc471cb70da216dcdee69a/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.642210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 809/1602] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:02.643248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01212c-1d4e-4867-a2a1-b4f0251ba505 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.643975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19f493-8082-486b-80af-2c0d29e57c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f0e8f52e794a398b6b5db5634cf60a, user_id=3dd72ff391dc471cb70da216dcdee69a, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.646757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19f493-8082-486b-80af-2c0d29e57c0c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.654868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01212c-1d4e-4867-a2a1-b4f0251ba505 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.659603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.662629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.662629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.662629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] PATH_INFO: `/v3/projects/a2a34e80c2fd4e0fb74d08649b7047a1/users/12ed53b2fa1c456e94a0083cff8bf62f/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.677761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a34e80c2fd4e0fb74d08649b7047a1, user_id=12ed53b2fa1c456e94a0083cff8bf62f, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.681168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.743736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 794/1603] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/50f0e8f52e794a398b6b5db5634cf60a/users/3dd72ff391dc471cb70da216dcdee69a/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.745139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ecfd6-ac74-4e94-bdf4-569f73f96889 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.745432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ecfd6-ac74-4e94-bdf4-569f73f96889 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.745715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ecfd6-ac74-4e94-bdf4-569f73f96889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.764285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 810/1604] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/a2a34e80c2fd4e0fb74d08649b7047a1/users/12ed53b2fa1c456e94a0083cff8bf62f/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.765178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.772642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4f3ecfd6-ac74-4e94-bdf4-569f73f96889 None None] MFA Rules not processed for user `95f0f1dea22d4870b8b117c3be6dcd5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:02.774547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4f3ecfd6-ac74-4e94-bdf4-569f73f96889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:02.827932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6c12d-3e80-47dc-ad8a-fe1351815f23 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.832557 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.833408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.833692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.834199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] PATH_INFO: `/v3/projects/c738995fef2342e99240de801574307d/users/5d079dc5e2b142c7a088648f79d1bb0b/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.839755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 795/1605] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:02.843575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ecfd6-ac74-4e94-bdf4-569f73f96889 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.850672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c738995fef2342e99240de801574307d, user_id=5d079dc5e2b142c7a088648f79d1bb0b, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.854153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.856442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ecfd6-ac74-4e94-bdf4-569f73f96889 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:02.861804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:02.862864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:02.863312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:02.863597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] PATH_INFO: `/v3/projects/492e3fa83b454f1185d99d89bcc1b1d1/users/c44abfcbae7142cc89847371cd7c0caa/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:02.879533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=492e3fa83b454f1185d99d89bcc1b1d1, user_id=c44abfcbae7142cc89847371cd7c0caa, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:02.882280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:02.926638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 811/1606] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/c738995fef2342e99240de801574307d/users/5d079dc5e2b142c7a088648f79d1bb0b/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.930098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.955433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 796/1607] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/492e3fa83b454f1185d99d89bcc1b1d1/users/c44abfcbae7142cc89847371cd7c0caa/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:02.956275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:02.976960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:02.988252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f547c3-ccd4-4d42-bbd8-f0cfb7f0e13c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.020421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650e6c-22dd-4b5b-90d3-93209ccacba9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.025162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a792a3-c3ff-4a08-be02-add8b2b792d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.026054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a792a3-c3ff-4a08-be02-add8b2b792d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.026341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a792a3-c3ff-4a08-be02-add8b2b792d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.026628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a792a3-c3ff-4a08-be02-add8b2b792d2 admin admin] PATH_INFO: `/v3/projects/e07f9778bf194321ac9279121c065a3d/users/fd3eef3ca9da4302a7c2e52acb8155c6/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.039320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:03.042363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a792a3-c3ff-4a08-be02-add8b2b792d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07f9778bf194321ac9279121c065a3d, user_id=fd3eef3ca9da4302a7c2e52acb8155c6, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.045155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a792a3-c3ff-4a08-be02-add8b2b792d2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.045913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d8949-6218-4a7a-abaf-80474f94ccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.047883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d8949-6218-4a7a-abaf-80474f94ccb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.047883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d8949-6218-4a7a-abaf-80474f94ccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.047883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d8949-6218-4a7a-abaf-80474f94ccb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.047883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d8949-6218-4a7a-abaf-80474f94ccb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.049653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d8949-6218-4a7a-abaf-80474f94ccb9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.059419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 812/1608] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:03.060669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925461-20a6-4745-915d-b2f07d5f5bdd None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.060965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925461-20a6-4745-915d-b2f07d5f5bdd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.061266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e925461-20a6-4745-915d-b2f07d5f5bdd None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.062777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 813/1609] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 11:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:03.063872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f629ae-d192-47e6-9b31-04e94b90f72e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.064161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f629ae-d192-47e6-9b31-04e94b90f72e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.064439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f629ae-d192-47e6-9b31-04e94b90f72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.082243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b8f629ae-d192-47e6-9b31-04e94b90f72e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.083435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b8f629ae-d192-47e6-9b31-04e94b90f72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.092091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 814/1610] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.099802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056e2ed-8ebc-4577-aeb8-c868c6946b3b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.099802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056e2ed-8ebc-4577-aeb8-c868c6946b3b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.099802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056e2ed-8ebc-4577-aeb8-c868c6946b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.119495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a056e2ed-8ebc-4577-aeb8-c868c6946b3b None None] MFA Rules not processed for user `12ed53b2fa1c456e94a0083cff8bf62f`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.120682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a056e2ed-8ebc-4577-aeb8-c868c6946b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.161039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 797/1611] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:02 2022] PUT /identity/v3/projects/e07f9778bf194321ac9279121c065a3d/users/fd3eef3ca9da4302a7c2e52acb8155c6/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:03.162401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4c91c-0cc1-485c-b1dd-74220fe19877 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.162688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4c91c-0cc1-485c-b1dd-74220fe19877 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.162964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4c91c-0cc1-485c-b1dd-74220fe19877 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.164484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 798/1612] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 11:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:03.165791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24138cfb-34d5-44b0-b918-0c68cfcada90 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.166148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24138cfb-34d5-44b0-b918-0c68cfcada90 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.166625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24138cfb-34d5-44b0-b918-0c68cfcada90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.183815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-24138cfb-34d5-44b0-b918-0c68cfcada90 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.185013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-24138cfb-34d5-44b0-b918-0c68cfcada90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.185395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 815/1613] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.186241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056e2ed-8ebc-4577-aeb8-c868c6946b3b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.200400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056e2ed-8ebc-4577-aeb8-c868c6946b3b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.246941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 799/1614] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.247809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24138cfb-34d5-44b0-b918-0c68cfcada90 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.256894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f11f-abfc-4eaa-93d7-41a8272da078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.258087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f11f-abfc-4eaa-93d7-41a8272da078 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.258375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f11f-abfc-4eaa-93d7-41a8272da078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.258650 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f11f-abfc-4eaa-93d7-41a8272da078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.259461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f11f-abfc-4eaa-93d7-41a8272da078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.260864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24138cfb-34d5-44b0-b918-0c68cfcada90 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.261101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f11f-abfc-4eaa-93d7-41a8272da078 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.265583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e847f5d-fce8-43d3-930b-e19f1fbaf53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.266607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e847f5d-fce8-43d3-930b-e19f1fbaf53d admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.266742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e847f5d-fce8-43d3-930b-e19f1fbaf53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.267024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e847f5d-fce8-43d3-930b-e19f1fbaf53d admin admin] PATH_INFO: `/v3/projects/3622ea9c334d40b08ca745f89be6b988/users/0fbb20d3c4bf4ed09a4e8c01ca10a927/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.271927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 816/1615] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:03.273722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46276568-23d4-405f-afc8-477c2832a369 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.274013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46276568-23d4-405f-afc8-477c2832a369 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.274290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46276568-23d4-405f-afc8-477c2832a369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.283781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e847f5d-fce8-43d3-930b-e19f1fbaf53d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3622ea9c334d40b08ca745f89be6b988, user_id=0fbb20d3c4bf4ed09a4e8c01ca10a927, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.286516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e847f5d-fce8-43d3-930b-e19f1fbaf53d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.292586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-46276568-23d4-405f-afc8-477c2832a369 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.293829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-46276568-23d4-405f-afc8-477c2832a369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.358617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 817/1616] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.359409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46276568-23d4-405f-afc8-477c2832a369 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.367759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 800/1617] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:03 2022] PUT /identity/v3/projects/3622ea9c334d40b08ca745f89be6b988/users/0fbb20d3c4bf4ed09a4e8c01ca10a927/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:03.367759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d62467-a528-410f-ae61-58c9a2648da3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.367759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d62467-a528-410f-ae61-58c9a2648da3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.367759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d62467-a528-410f-ae61-58c9a2648da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.384566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-11d62467-a528-410f-ae61-58c9a2648da3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.386904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-11d62467-a528-410f-ae61-58c9a2648da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.418267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46276568-23d4-405f-afc8-477c2832a369 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.446773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 801/1618] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.448228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08c22c-a2f8-45b5-be4a-41b0ae3f2748 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.448536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08c22c-a2f8-45b5-be4a-41b0ae3f2748 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.448821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08c22c-a2f8-45b5-be4a-41b0ae3f2748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.451417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:03.466860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8e08c22c-a2f8-45b5-be4a-41b0ae3f2748 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.468136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8e08c22c-a2f8-45b5-be4a-41b0ae3f2748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.477144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.477892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.478758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.479034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.480963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:03.488375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.489857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.500381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 818/1619] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.501194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.541147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 802/1620] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.542009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08c22c-a2f8-45b5-be4a-41b0ae3f2748 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.550565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08c22c-a2f8-45b5-be4a-41b0ae3f2748 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.554970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.555650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.555939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.556225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.557743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11590e96-b1a3-4c17-81c4-3325d2097932 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.561406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.562162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.562448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.562723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.568859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.570891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.583162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 803/1621] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.583162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.592705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c90c01-77c2-4577-928c-26283aef4df8 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.598875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b48f0-b6a8-4c36-90d9-37b0772e5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.599901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b48f0-b6a8-4c36-90d9-37b0772e5aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.600187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b48f0-b6a8-4c36-90d9-37b0772e5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.600476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b48f0-b6a8-4c36-90d9-37b0772e5aef admin admin] PATH_INFO: `/v3/projects/492e3fa83b454f1185d99d89bcc1b1d1/users/c44abfcbae7142cc89847371cd7c0caa/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.602842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b48f0-b6a8-4c36-90d9-37b0772e5aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=492e3fa83b454f1185d99d89bcc1b1d1, user_id=c44abfcbae7142cc89847371cd7c0caa, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.605603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b48f0-b6a8-4c36-90d9-37b0772e5aef admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.627967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 804/1622] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:03 2022] PUT /identity/v3/projects/492e3fa83b454f1185d99d89bcc1b1d1/users/c44abfcbae7142cc89847371cd7c0caa/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:03.627967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d596c-f015-4fa6-ae8f-6b2d2ad478b3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.627967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d596c-f015-4fa6-ae8f-6b2d2ad478b3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.627967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d596c-f015-4fa6-ae8f-6b2d2ad478b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.637559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.639254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.648870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1a3d596c-f015-4fa6-ae8f-6b2d2ad478b3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.649440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 819/1623] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.650335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.652270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1a3d596c-f015-4fa6-ae8f-6b2d2ad478b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.663760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d61587-715a-4f2c-b0e1-58520968bc12 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.677788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:03.722387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 805/1624] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:03.722387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d596c-f015-4fa6-ae8f-6b2d2ad478b3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.751993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1409be-9fbb-4ccd-a237-a6d64d314136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.752995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1409be-9fbb-4ccd-a237-a6d64d314136 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.753502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1409be-9fbb-4ccd-a237-a6d64d314136 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.753987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1409be-9fbb-4ccd-a237-a6d64d314136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.794160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d596c-f015-4fa6-ae8f-6b2d2ad478b3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.798249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.798912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.799275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.799558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.815562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.815562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.822412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 806/1625] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.823270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.834251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1409be-9fbb-4ccd-a237-a6d64d314136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.835110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5397f7-5cb8-4766-999f-59ae5d81e4e9 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:03.836949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1409be-9fbb-4ccd-a237-a6d64d314136 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.842212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:03.856324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.856324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.856324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.856689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 820/1626] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.856689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3af2d-1066-482e-80e5-ddd5802ee2c2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:03.856689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3af2d-1066-482e-80e5-ddd5802ee2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:03.856689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3af2d-1066-482e-80e5-ddd5802ee2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:03.873019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:03.921945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a0a3af2d-1066-482e-80e5-ddd5802ee2c2 None None] MFA Rules not processed for user `c44abfcbae7142cc89847371cd7c0caa`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:03.926648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a0a3af2d-1066-482e-80e5-ddd5802ee2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:03.938943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:03.945616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:03.972693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 807/1627] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:03.977197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:03.981209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:04.005348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868438c-9efd-4fc2-9a5b-ff2c2144302f service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:04.010385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:04.011210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.011582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.011930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.022617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:04.024338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:04.047827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 808/1628] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:04.070089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 821/1629] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:03 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:04.079573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:04.231263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fea907-d1f0-4fdd-ac0a-28ffeb1dc904 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.231562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fea907-d1f0-4fdd-ac0a-28ffeb1dc904 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.232131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fea907-d1f0-4fdd-ac0a-28ffeb1dc904 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.233505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 822/1630] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 11:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:10:04.239078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66eaf3-5af5-4379-a897-d91aa378dfd3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.239396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66eaf3-5af5-4379-a897-d91aa378dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.239687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66eaf3-5af5-4379-a897-d91aa378dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.259753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41051fcb-6745-4375-af04-d657c7d5f3d2 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:04.265651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2f66eaf3-5af5-4379-a897-d91aa378dfd3 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:04.267133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2f66eaf3-5af5-4379-a897-d91aa378dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:04.276788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:04.281944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.282283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.282575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.286597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 823/1631] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:10:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:04.294400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d4e4-aa35-40bd-848f-06d199a26426 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.299084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d4e4-aa35-40bd-848f-06d199a26426 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.299578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3d4e4-aa35-40bd-848f-06d199a26426 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.301518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 824/1632] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 11:10:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:10:04.310809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25aa5d-bb39-4cae-bdc7-d65b44a0aa68 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.310809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25aa5d-bb39-4cae-bdc7-d65b44a0aa68 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.310809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25aa5d-bb39-4cae-bdc7-d65b44a0aa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.315520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:04.316982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:04.339282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 809/1633] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:04 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:04.343038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ee25aa5d-bb39-4cae-bdc7-d65b44a0aa68 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:04.344427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ee25aa5d-bb39-4cae-bdc7-d65b44a0aa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:04.355118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 825/1634] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:10:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:04.507978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:04.519970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b095ce-70c2-42b1-ae8e-294d70ae119e service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:04.529426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:04.530223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.533681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.533681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.552566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:04.556578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:04.573306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 810/1635] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:04.661606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25aa5d-bb39-4cae-bdc7-d65b44a0aa68 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:04.671602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25aa5d-bb39-4cae-bdc7-d65b44a0aa68 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:04.678269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:04.678864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:04.679356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:04.679652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:04.695934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:04.697870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:04.721372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 826/1636] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:05.069688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:05.080357 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eebbaa-d994-4d13-ae90-7a8424e0e56f service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:05.087851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e646c7-2a0b-4ba3-aed6-0826eaf5e3dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:05.088522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e646c7-2a0b-4ba3-aed6-0826eaf5e3dc service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:05.089395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e646c7-2a0b-4ba3-aed6-0826eaf5e3dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:05.089688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e646c7-2a0b-4ba3-aed6-0826eaf5e3dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:05.188729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e646c7-2a0b-4ba3-aed6-0826eaf5e3dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:05.190561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e646c7-2a0b-4ba3-aed6-0826eaf5e3dc service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:05.204202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 811/1637] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:05.988464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:06.009165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fd029-7a9b-41ab-a3ce-53b83103785f service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:06.014427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:06.015359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:06.015669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:06.023756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] PATH_INFO: `/v3/users/5e7ec74968f149f78334f707f7ff501a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:06.023756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7ec74968f149f78334f707f7ff501a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:06.023756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:06.042657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:06.350053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7ec74968f149f78334f707f7ff501a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:06.350053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7ec74968f149f78334f707f7ff501a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:06.350053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7ec74968f149f78334f707f7ff501a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:06.350053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7ec74968f149f78334f707f7ff501a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:06.350053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7ec74968f149f78334f707f7ff501a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:06.469740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 827/1638] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:05 2022] DELETE /identity/v3/users/5e7ec74968f149f78334f707f7ff501a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:06.474402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:06.474878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:06.475009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:06.501896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:06.503122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:06.562466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:06.599272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 812/1639] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:06.610528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:06.612268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:06.705274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7b292-5821-4fbc-96fb-fbe945d4a699 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:06.709691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d41a92-3af8-418c-a855-b3f2c793c07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:06.710693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41a92-3af8-418c-a855-b3f2c793c07a service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:06.711332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41a92-3af8-418c-a855-b3f2c793c07a service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:06.711773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41a92-3af8-418c-a855-b3f2c793c07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:06.724742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d41a92-3af8-418c-a855-b3f2c793c07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:06.726824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d41a92-3af8-418c-a855-b3f2c793c07a service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:06.738708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 828/1640] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:07.910086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:07.912521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:07.933251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e4fb8-ed90-445c-bc97-88a070ffef6d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:07.944917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:07.953583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:07.953583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:07.953583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] PATH_INFO: `/v3/projects/515a0335f2e04273b6efd97405f6c7b8` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:07.953583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=515a0335f2e04273b6efd97405f6c7b8)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:07.955942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:08.157571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515a0335f2e04273b6efd97405f6c7b8', 'request_id': 'req-71c06162-afed-473f-8019-89f0b28ae0b4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:08.166914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515a0335f2e04273b6efd97405f6c7b8', 'request_id': 'req-71c06162-afed-473f-8019-89f0b28ae0b4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:08.176579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515a0335f2e04273b6efd97405f6c7b8', 'request_id': 'req-71c06162-afed-473f-8019-89f0b28ae0b4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:08.285301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] The token cache is being invalidate because project 515a0335f2e04273b6efd97405f6c7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:08.290166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:08.290166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-71c06162-afed-473f-8019-89f0b28ae0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:08.295166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 813/1641] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:07 2022] DELETE /identity/v3/projects/515a0335f2e04273b6efd97405f6c7b8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:08.297412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801751d1-cd8e-405b-95c1-497cc0d3823b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:08.298876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801751d1-cd8e-405b-95c1-497cc0d3823b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:08.299406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801751d1-cd8e-405b-95c1-497cc0d3823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:08.326515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-801751d1-cd8e-405b-95c1-497cc0d3823b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:08.327918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-801751d1-cd8e-405b-95c1-497cc0d3823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:08.489812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 829/1642] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:08.510971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9388c79-bbf0-47e8-a3df-d7254d575450 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:08.513575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9388c79-bbf0-47e8-a3df-d7254d575450 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:08.513873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9388c79-bbf0-47e8-a3df-d7254d575450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:08.581015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f9388c79-bbf0-47e8-a3df-d7254d575450 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:08.585804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f9388c79-bbf0-47e8-a3df-d7254d575450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:08.604522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 814/1643] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:08.605790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801751d1-cd8e-405b-95c1-497cc0d3823b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:08.639604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801751d1-cd8e-405b-95c1-497cc0d3823b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:08.655780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:08.665814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:08.665814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:08.665814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:08.665814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:08.665814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:08.676692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:08.721986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 830/1644] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:08.734775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9388c79-bbf0-47e8-a3df-d7254d575450 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:08.748689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9388c79-bbf0-47e8-a3df-d7254d575450 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:08.775872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:08.778189 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:08.784839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:08.784839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:08.784839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:08.789891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:08.935821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 815/1645] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:08 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:08.940335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:08.981721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e2b5b-6465-4ba8-a62c-8e85d732afe1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:09.034649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:09.248756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:09.249491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:09.249836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:09.250110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:09.250831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:09.252847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:09.392848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 831/1646] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:08 2022] POST /identity/v3/users => generated 291 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:09.393903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:09.413292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3bfb8-3cff-4755-bdd5-04b4f4d53ec9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:09.425447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:09.426592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:09.427134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:09.428290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:09.430313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:09.432378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:09.446188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 816/1647] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:10:09.450250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:09.462741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55935-602b-4414-9c46-97ed9a74d97a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:09.476895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee5a07-db64-401a-8efe-8e8df5655aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:09.478629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee5a07-db64-401a-8efe-8e8df5655aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:09.479430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee5a07-db64-401a-8efe-8e8df5655aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:09.480474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee5a07-db64-401a-8efe-8e8df5655aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:09.482462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee5a07-db64-401a-8efe-8e8df5655aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:09.487629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee5a07-db64-401a-8efe-8e8df5655aa8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:09.504122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 832/1648] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:09.509207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:09.527054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92e18e-162c-4a84-b974-ffe6b2a88cff admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:09.535126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:09.536317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:09.537130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:09.537619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] PATH_INFO: `/v3/projects/668531e7995f41c984e0ca9b1f3a0c92/users/d37777b81a8e49898b56eafbb2cb2f28/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:09.556983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=668531e7995f41c984e0ca9b1f3a0c92, user_id=d37777b81a8e49898b56eafbb2cb2f28, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:09.565602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:09.660726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 817/1649] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:09 2022] PUT /identity/v3/projects/668531e7995f41c984e0ca9b1f3a0c92/users/d37777b81a8e49898b56eafbb2cb2f28/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:09.676450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404faf6-c03d-4323-835d-00e8be8a3d6f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:09.676821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404faf6-c03d-4323-835d-00e8be8a3d6f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:09.677285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404faf6-c03d-4323-835d-00e8be8a3d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:09.738328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e404faf6-c03d-4323-835d-00e8be8a3d6f None None] MFA Rules not processed for user `d37777b81a8e49898b56eafbb2cb2f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:09.740713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e404faf6-c03d-4323-835d-00e8be8a3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:09.834179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 833/1650] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:09.838655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:09.962037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bce299-8d34-4fc6-b37c-f6c596297f1f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:09.968266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:09.969447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:09.969916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:09.970336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:09.971478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:09.973751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:09.978750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:10.108773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 818/1651] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:09 2022] POST /identity/v3/projects => generated 379 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:10.112283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404faf6-c03d-4323-835d-00e8be8a3d6f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:10.126997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404faf6-c03d-4323-835d-00e8be8a3d6f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:10.231135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:10.233227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:10.233614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:10.233902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:10.236234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:10.238580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:10.371354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 834/1652] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:10 2022] POST /identity/v3/users => generated 296 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:10.375359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:10.401174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952706a-db52-4a21-aab4-2d46719c65a2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:10.410373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:10.411085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:10.411686 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:10.411976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:10.414655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:10.418587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:10.446205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 819/1653] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:10.450826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:10.467435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ccd5b-0a89-423a-b369-8a0d0f36a2ff admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:10.479880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:10.481062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:10.481512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:10.481951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] PATH_INFO: `/v3/projects/7430841f6e7a456ab00d76ab414d8549/users/48973709b70d4042a26c04917c445ac4/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:10.519045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7430841f6e7a456ab00d76ab414d8549, user_id=48973709b70d4042a26c04917c445ac4, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:10.523917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:10.635499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 835/1654] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:10 2022] PUT /identity/v3/projects/7430841f6e7a456ab00d76ab414d8549/users/48973709b70d4042a26c04917c445ac4/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:10.637181 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:10.654576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d743de0-71dd-4b89-a2c6-9dd9e0939fea admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:10.775671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:10.822648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a9845-892e-4dcf-8a57-d6135faffe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:10.823411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a9845-892e-4dcf-8a57-d6135faffe02 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:10.823707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a9845-892e-4dcf-8a57-d6135faffe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:10.823989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a9845-892e-4dcf-8a57-d6135faffe02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:10.831023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a9845-892e-4dcf-8a57-d6135faffe02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:10.832970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a9845-892e-4dcf-8a57-d6135faffe02 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:10.876388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 820/1655] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:10.880590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:10.900022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28b1f-3b05-400e-b228-a005703a99cf admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:10.922280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:10.923352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:10.923813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:10.924256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] PATH_INFO: `/v3/projects/7430841f6e7a456ab00d76ab414d8549/users/48973709b70d4042a26c04917c445ac4/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:10.929534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7430841f6e7a456ab00d76ab414d8549, user_id=48973709b70d4042a26c04917c445ac4, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:10.937055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:10.970440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 836/1656] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:10 2022] PUT /identity/v3/projects/7430841f6e7a456ab00d76ab414d8549/users/48973709b70d4042a26c04917c445ac4/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:10.980812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:10.982031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:10.983746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:11.049573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] MFA Rules not processed for user `48973709b70d4042a26c04917c445ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:11.050759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:11.230632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 821/1657] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:11.255149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:11.257534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:11.417439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd784fa-6fad-4c16-a395-0d6d1d51726c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:11.430667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cc03a-7a94-4e56-8b7f-da21f1c9d3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:11.431418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc03a-7a94-4e56-8b7f-da21f1c9d3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:11.431714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc03a-7a94-4e56-8b7f-da21f1c9d3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:11.432118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc03a-7a94-4e56-8b7f-da21f1c9d3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:11.454294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cc03a-7a94-4e56-8b7f-da21f1c9d3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:11.457117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cc03a-7a94-4e56-8b7f-da21f1c9d3a7 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:11.485169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 837/1658] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:11.875460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:11.876951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:11.895810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04555f14-ad98-4fe2-bb03-77e427c2891c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:11.907881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979f383-8891-4ac7-bd47-cdb39c06ec45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:11.908591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979f383-8891-4ac7-bd47-cdb39c06ec45 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:11.909812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979f383-8891-4ac7-bd47-cdb39c06ec45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:11.910256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979f383-8891-4ac7-bd47-cdb39c06ec45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:12.041635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:12.077023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979f383-8891-4ac7-bd47-cdb39c06ec45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:12.078713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979f383-8891-4ac7-bd47-cdb39c06ec45 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:12.106988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 822/1659] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:14.744351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:14.744914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:14.745029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:14.801516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] MFA Rules not processed for user `0fbb20d3c4bf4ed09a4e8c01ca10a927`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:14.808164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:14.895224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 838/1660] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:14.922106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca6c78-ae2c-4188-89da-75d71d15e818 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:14.924058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca6c78-ae2c-4188-89da-75d71d15e818 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:14.925819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca6c78-ae2c-4188-89da-75d71d15e818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:14.992040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a9ca6c78-ae2c-4188-89da-75d71d15e818 None None] MFA Rules not processed for user `0fbb20d3c4bf4ed09a4e8c01ca10a927`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:14.994666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a9ca6c78-ae2c-4188-89da-75d71d15e818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:15.005508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 823/1661] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:15.021493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:15.024603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:15.194792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:15.203753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0ac98-df5b-49c3-b0b7-c2332707e7ad None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:15.227865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bfdcf-4473-4989-ae2a-cc7315d98adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:15.228572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bfdcf-4473-4989-ae2a-cc7315d98adc service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:15.228865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bfdcf-4473-4989-ae2a-cc7315d98adc service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:15.229148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bfdcf-4473-4989-ae2a-cc7315d98adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:15.253061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bfdcf-4473-4989-ae2a-cc7315d98adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:15.257818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bfdcf-4473-4989-ae2a-cc7315d98adc service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:15.279426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 839/1662] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:16.222530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ad820-c26d-4aba-80f9-1b48303d0726 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:16.225067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ad820-c26d-4aba-80f9-1b48303d0726 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:16.225067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ad820-c26d-4aba-80f9-1b48303d0726 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:16.230404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 824/1663] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 11:10:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:16.231627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bc57-5a1c-445e-b158-0fd08008a3cc None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:16.232119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bc57-5a1c-445e-b158-0fd08008a3cc None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:16.232119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bc57-5a1c-445e-b158-0fd08008a3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:16.308294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3c96bc57-5a1c-445e-b158-0fd08008a3cc None None] MFA Rules not processed for user `cf08ecfc3fed44e2bc1ccff8521a4ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:16.309703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3c96bc57-5a1c-445e-b158-0fd08008a3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:16.403872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 840/1664] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:10:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:16.433190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ad820-c26d-4aba-80f9-1b48303d0726 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:16.434916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5d5ad820-c26d-4aba-80f9-1b48303d0726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:16.473585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ad820-c26d-4aba-80f9-1b48303d0726 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:16.496669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52305a85-6042-4fd3-b4e8-38afb68ebbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:16.496669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52305a85-6042-4fd3-b4e8-38afb68ebbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:16.496669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52305a85-6042-4fd3-b4e8-38afb68ebbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:16.496669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52305a85-6042-4fd3-b4e8-38afb68ebbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:16.521317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52305a85-6042-4fd3-b4e8-38afb68ebbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:16.523371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52305a85-6042-4fd3-b4e8-38afb68ebbb5 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:16.542340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 825/1665] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:17.742679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9eb2f-2c2a-4ccc-a20e-ade3416fe39a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:17.743240 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9eb2f-2c2a-4ccc-a20e-ade3416fe39a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:17.743332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9eb2f-2c2a-4ccc-a20e-ade3416fe39a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:17.745440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 841/1666] 10.222.0.43 () {54 vars in 960 bytes} [Thu Jun 23 11:10:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:17.755797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2649-35e4-49e9-935f-afe9f56fddd3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:17.756290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2649-35e4-49e9-935f-afe9f56fddd3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:17.756555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2649-35e4-49e9-935f-afe9f56fddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:17.818995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-aade2649-35e4-49e9-935f-afe9f56fddd3 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:17.822366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-aade2649-35e4-49e9-935f-afe9f56fddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:17.887798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9305d-2f2c-42c9-946d-9015d3577207 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:17.888095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9305d-2f2c-42c9-946d-9015d3577207 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:17.888386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9305d-2f2c-42c9-946d-9015d3577207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:17.934693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3bf9305d-2f2c-42c9-946d-9015d3577207 None None] MFA Rules not processed for user `0fbb20d3c4bf4ed09a4e8c01ca10a927`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:17.940497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3bf9305d-2f2c-42c9-946d-9015d3577207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:17.962393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 842/1667] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:18.053933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9305d-2f2c-42c9-946d-9015d3577207 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:18.066552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 826/1668] 10.222.0.43 () {58 vars in 1057 bytes} [Thu Jun 23 11:10:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:18.073461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9305d-2f2c-42c9-946d-9015d3577207 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:18.083545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec2aa0-4683-463d-841e-ba2f63120685 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:18.086010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec2aa0-4683-463d-841e-ba2f63120685 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:18.087454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec2aa0-4683-463d-841e-ba2f63120685 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:18.087882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec2aa0-4683-463d-841e-ba2f63120685 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:18.092592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1aec2aa0-4683-463d-841e-ba2f63120685 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:18.156042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec2aa0-4683-463d-841e-ba2f63120685 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:18.156042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec2aa0-4683-463d-841e-ba2f63120685 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:18.186262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 843/1669] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:18.801809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5adb-f5f4-484f-bcc8-17f48ccc91b9 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:18.801809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5adb-f5f4-484f-bcc8-17f48ccc91b9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:18.801809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5adb-f5f4-484f-bcc8-17f48ccc91b9 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:18.803151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 827/1670] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 11:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:18.813552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f57b91-b283-4885-bd38-2372456d4094 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:18.814491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f57b91-b283-4885-bd38-2372456d4094 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:18.814953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f57b91-b283-4885-bd38-2372456d4094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:18.862375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-62f57b91-b283-4885-bd38-2372456d4094 None None] MFA Rules not processed for user `cf08ecfc3fed44e2bc1ccff8521a4ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:18.865382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-62f57b91-b283-4885-bd38-2372456d4094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:18.903748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 844/1671] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:10:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:18.905499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e5adb-f5f4-484f-bcc8-17f48ccc91b9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:18.927385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e5adb-f5f4-484f-bcc8-17f48ccc91b9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:18.937657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:18.938574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:18.939032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:18.939629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:18.966518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:18.968268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:18.992415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 828/1672] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:19.241623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b14a18-3a2b-496a-832a-0c5ce262ecb8 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:19.241623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b14a18-3a2b-496a-832a-0c5ce262ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:19.241623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b14a18-3a2b-496a-832a-0c5ce262ecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:19.328656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-72b14a18-3a2b-496a-832a-0c5ce262ecb8 None None] MFA Rules not processed for user `fd3eef3ca9da4302a7c2e52acb8155c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:19.330071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-72b14a18-3a2b-496a-832a-0c5ce262ecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:19.501144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 845/1673] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:19.745947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:19.747612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:19.757444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b14a18-3a2b-496a-832a-0c5ce262ecb8 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:19.782794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b14a18-3a2b-496a-832a-0c5ce262ecb8 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:19.801473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb054a-9b67-4d11-9852-f9d78779958c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:19.802145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb054a-9b67-4d11-9852-f9d78779958c service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:19.802431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb054a-9b67-4d11-9852-f9d78779958c service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:19.802707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb054a-9b67-4d11-9852-f9d78779958c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:19.831359 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb054a-9b67-4d11-9852-f9d78779958c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:19.839462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb054a-9b67-4d11-9852-f9d78779958c service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:19.872669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 846/1674] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:19.934436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53fccf-67dc-4ed8-b507-a15703821e39 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:19.947625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:19.948578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:19.948804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:19.951340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] PATH_INFO: `/v3/users/95f0f1dea22d4870b8b117c3be6dcd5a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:19.953125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f0f1dea22d4870b8b117c3be6dcd5a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:19.955347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: WARNING py.warnings [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:10:19.955347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: warnings.warn(msg) Jun 23 11:10:19.955347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: Jun 23 11:10:19.958950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:20.285712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f0f1dea22d4870b8b117c3be6dcd5a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:20.286680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f0f1dea22d4870b8b117c3be6dcd5a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:20.309638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f0f1dea22d4870b8b117c3be6dcd5a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:20.311319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f0f1dea22d4870b8b117c3be6dcd5a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:20.329462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:20.429767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf79a3-0b13-4902-a36c-49eb781fa06e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:20.430134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf79a3-0b13-4902-a36c-49eb781fa06e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:20.430413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf79a3-0b13-4902-a36c-49eb781fa06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:20.432249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ce709617-db7e-4f5f-8024-3cbc34204d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f0f1dea22d4870b8b117c3be6dcd5a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:20.533362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 829/1675] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:19 2022] DELETE /identity/v3/users/95f0f1dea22d4870b8b117c3be6dcd5a => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:20.535225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:20.535534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:20.536077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:20.537109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-60cf79a3-0b13-4902-a36c-49eb781fa06e None None] MFA Rules not processed for user `3dd72ff391dc471cb70da216dcdee69a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:20.539146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-60cf79a3-0b13-4902-a36c-49eb781fa06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:20.563300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:20.565693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:20.604745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:20.705461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 830/1676] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:20.730741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:20.732633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:20.755394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 847/1677] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:20.762753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:20.763540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:20.763892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:20.790348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] MFA Rules not processed for user `3dd72ff391dc471cb70da216dcdee69a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:20.792472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:20.807890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 848/1678] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:20.815025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:20.819884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:20.838502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e1f2-85d2-4fe7-a79c-129e7dbbb8cb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:20.847466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7dbac-9838-4e7a-9e4a-20ecc2d0ae56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:20.848365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7dbac-9838-4e7a-9e4a-20ecc2d0ae56 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:20.849022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7dbac-9838-4e7a-9e4a-20ecc2d0ae56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:20.849632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7dbac-9838-4e7a-9e4a-20ecc2d0ae56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:20.880220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7dbac-9838-4e7a-9e4a-20ecc2d0ae56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:20.881997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7dbac-9838-4e7a-9e4a-20ecc2d0ae56 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:20.931687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 831/1679] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:21.097957 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fc9a7-cd57-4be6-be8f-52b8f3b90e86 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:21.106638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:21.107482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:21.109286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:21.109745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:21.135240 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:21.137671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:21.189957 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 849/1680] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:21.656767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:21.657334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:21.657334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:21.721762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] MFA Rules not processed for user `5d079dc5e2b142c7a088648f79d1bb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:21.723027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:21.894457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:21.896346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:22.044344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2760535-f260-4f2e-9b7f-8f64bef4a7b9 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:22.068904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 832/1681] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:22.074709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:22.080343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:22.080996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:22.081909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:22.089425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:22.090415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] PATH_INFO: `/v3/projects/03469431b9b64579b92f8714a35397c7` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:22.094577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03469431b9b64579b92f8714a35397c7)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:22.103162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: WARNING py.warnings [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:10:22.103162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: warnings.warn(msg) Jun 23 11:10:22.103162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: Jun 23 11:10:22.109122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9944d-c2e0-4e4b-be37-fd07e096ba6d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:22.115957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7bf3d-4325-4d8e-a88f-4700582f1131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:22.116902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7bf3d-4325-4d8e-a88f-4700582f1131 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:22.117409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7bf3d-4325-4d8e-a88f-4700582f1131 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:22.117519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:22.117926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7bf3d-4325-4d8e-a88f-4700582f1131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:22.131942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7bf3d-4325-4d8e-a88f-4700582f1131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:22.134066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7bf3d-4325-4d8e-a88f-4700582f1131 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:22.149019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 833/1682] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:22.440352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03469431b9b64579b92f8714a35397c7', 'request_id': 'req-b46860c8-d456-4269-812c-af4dd1e5fad0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:22.475133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:22.475853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03469431b9b64579b92f8714a35397c7', 'request_id': 'req-b46860c8-d456-4269-812c-af4dd1e5fad0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:22.506366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03469431b9b64579b92f8714a35397c7', 'request_id': 'req-b46860c8-d456-4269-812c-af4dd1e5fad0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:22.672529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] The token cache is being invalidate because project 03469431b9b64579b92f8714a35397c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:22.672833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:22.675637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b46860c8-d456-4269-812c-af4dd1e5fad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:22.699252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 850/1683] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:21 2022] DELETE /identity/v3/projects/03469431b9b64579b92f8714a35397c7 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:22.701063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3437cfc-90bf-4142-b329-3acd9c03246c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:22.701558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3437cfc-90bf-4142-b329-3acd9c03246c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:22.701848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3437cfc-90bf-4142-b329-3acd9c03246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:22.749311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d3437cfc-90bf-4142-b329-3acd9c03246c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:22.753635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d3437cfc-90bf-4142-b329-3acd9c03246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:22.953775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 834/1684] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:22.962960 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e3e53-931f-4fc2-8337-cecc70900db2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:22.963459 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e3e53-931f-4fc2-8337-cecc70900db2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:22.963888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e3e53-931f-4fc2-8337-cecc70900db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:23.004832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9e5e3e53-931f-4fc2-8337-cecc70900db2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:23.006693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9e5e3e53-931f-4fc2-8337-cecc70900db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:23.027727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 851/1685] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:23.033401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3437cfc-90bf-4142-b329-3acd9c03246c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:23.051599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3437cfc-90bf-4142-b329-3acd9c03246c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:23.080599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:23.081305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:23.081597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:23.081896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:23.082653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:23.082966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:23.090700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:23.113867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 835/1686] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:23.118468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e3e53-931f-4fc2-8337-cecc70900db2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:23.133986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e3e53-931f-4fc2-8337-cecc70900db2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:23.139345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:23.139965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:23.140299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:23.140747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:23.141622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:23.143983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:23.284020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:23.294593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 852/1687] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:23 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:23.298900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:23.312869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d5a8c-c197-4e79-bad4-0162429a84f9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:23.475689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:23.476544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:23.476841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:23.477127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:23.477868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:23.492150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:23.704426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 836/1688] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:23 2022] POST /identity/v3/users => generated 286 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:23.708401 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:23.729197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c193d0-5959-4811-a928-5fa355e68ab8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:23.751337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:23.752111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:23.752588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:23.753188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:23.756045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:23.764151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:23.796529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 853/1689] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:23.801399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:23.812800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f49ef-8a9e-44c0-9a23-329ce12179d1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:23.822324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9fd5-e60a-407e-b300-1ea7aa6f3fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:23.823025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9fd5-e60a-407e-b300-1ea7aa6f3fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:23.823355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9fd5-e60a-407e-b300-1ea7aa6f3fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:23.823632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9fd5-e60a-407e-b300-1ea7aa6f3fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:23.825757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9fd5-e60a-407e-b300-1ea7aa6f3fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:23.832554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9fd5-e60a-407e-b300-1ea7aa6f3fbe admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:23.849692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 837/1690] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:23.854230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:23.867301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636ac26-828e-4fab-bf74-ede905110171 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:23.899916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:23.901237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:23.901717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:23.902213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] PATH_INFO: `/v3/projects/924a0f8fe1da49eb9bc3ddac4e09419f/users/faebd8955e2348b8b54456f0c3c9b00c/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:23.944759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924a0f8fe1da49eb9bc3ddac4e09419f, user_id=faebd8955e2348b8b54456f0c3c9b00c, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:23.948871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:24.175288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 854/1691] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:23 2022] PUT /identity/v3/projects/924a0f8fe1da49eb9bc3ddac4e09419f/users/faebd8955e2348b8b54456f0c3c9b00c/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:24.190247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f0113-a3ac-4353-bfd2-690bc7ee7f85 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:24.190936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f0113-a3ac-4353-bfd2-690bc7ee7f85 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:24.191456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f0113-a3ac-4353-bfd2-690bc7ee7f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:24.276979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-235f0113-a3ac-4353-bfd2-690bc7ee7f85 None None] MFA Rules not processed for user `faebd8955e2348b8b54456f0c3c9b00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:24.276979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-235f0113-a3ac-4353-bfd2-690bc7ee7f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:24.478947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:24.555711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 838/1692] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:24.557395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:24.799634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16dcef-4c8b-4025-80fd-63732e50e2f5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:24.817590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:24.820185 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:24.824298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:24.824298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:24.824298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:24.832641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:25.086669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 855/1693] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:24 2022] POST /identity/v3/projects => generated 371 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:25.088859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f0113-a3ac-4353-bfd2-690bc7ee7f85 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:25.129493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f0113-a3ac-4353-bfd2-690bc7ee7f85 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:25.338398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:25.339100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:25.339418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:25.339707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:25.340433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:25.342552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:25.485984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 839/1694] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:25 2022] POST /identity/v3/users => generated 292 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:25.486431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:25.509312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9eb743-c997-4778-8938-ea198691ce7a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:25.532812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36735b4-2c16-4850-9db2-345e14470e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:25.534455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36735b4-2c16-4850-9db2-345e14470e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:25.537985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36735b4-2c16-4850-9db2-345e14470e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:25.538683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36735b4-2c16-4850-9db2-345e14470e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:25.541386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36735b4-2c16-4850-9db2-345e14470e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:25.552426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36735b4-2c16-4850-9db2-345e14470e49 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:25.581582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 856/1695] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:10:25.585866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:25.613073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9077143-26f9-41af-8b10-1393bf981e17 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:25.631979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:25.637844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:25.638143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:25.638449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] PATH_INFO: `/v3/projects/8616647d0b184e1ea67c1368c48cf69c/users/4ffc5a0054cf4ff7990080439d3ed933/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:25.669802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d967c5d-9fd5-4c3a-bb6f-4c867db1b30e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:25.670367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d967c5d-9fd5-4c3a-bb6f-4c867db1b30e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:25.670674 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d967c5d-9fd5-4c3a-bb6f-4c867db1b30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:25.670784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8616647d0b184e1ea67c1368c48cf69c, user_id=4ffc5a0054cf4ff7990080439d3ed933, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:25.678609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:25.733830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6d967c5d-9fd5-4c3a-bb6f-4c867db1b30e None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:25.736315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6d967c5d-9fd5-4c3a-bb6f-4c867db1b30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:25.854407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 840/1696] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:25 2022] PUT /identity/v3/projects/8616647d0b184e1ea67c1368c48cf69c/users/4ffc5a0054cf4ff7990080439d3ed933/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:25.859547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:25.874013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 857/1697] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:10:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:26.071977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e3444-ec26-453b-aa78-d0fd4f300983 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:26.083259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3e3c5-988f-4ea4-9363-3f7e2989c606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:26.085934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e3c5-988f-4ea4-9363-3f7e2989c606 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:26.086447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e3c5-988f-4ea4-9363-3f7e2989c606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:26.086954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e3c5-988f-4ea4-9363-3f7e2989c606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:26.089565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3e3c5-988f-4ea4-9363-3f7e2989c606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:26.092922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3e3c5-988f-4ea4-9363-3f7e2989c606 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:26.129967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 841/1698] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:26.133884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d967c5d-9fd5-4c3a-bb6f-4c867db1b30e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:26.153017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d967c5d-9fd5-4c3a-bb6f-4c867db1b30e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:26.172547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:26.173538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:26.173900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:26.174259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] PATH_INFO: `/v3/projects/8616647d0b184e1ea67c1368c48cf69c/users/4ffc5a0054cf4ff7990080439d3ed933/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:26.181590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8616647d0b184e1ea67c1368c48cf69c, user_id=4ffc5a0054cf4ff7990080439d3ed933, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:26.184426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:26.219397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 858/1699] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:26 2022] PUT /identity/v3/projects/8616647d0b184e1ea67c1368c48cf69c/users/4ffc5a0054cf4ff7990080439d3ed933/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:26.221387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:26.221755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:26.222112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:26.270845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] MFA Rules not processed for user `4ffc5a0054cf4ff7990080439d3ed933`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:26.272234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:26.517824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 842/1700] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:26.528382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:26.531246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:26.574489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:26.576257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:26.707396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47287c2b-ff58-4e23-810f-7f500bd62784 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:26.721363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:26.722136 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:26.722551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:26.722903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:26.743896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:26.749273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:26.773957 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 859/1701] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:26.789954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:26.801186 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:26.812395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae572a5-f1e2-4233-bfea-a596eee53e91 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:26.829193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:26.830564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:26.832386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:26.832600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:26.918654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:26.920338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:26.921194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:26.930596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 843/1702] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:10:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 11:10:26.931485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:26.934406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:27.021401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:27.073470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859c6d-c4ff-4fa1-83e9-f1ca1a494d0a service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:27.093104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:27.094015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:27.094456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:27.099005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] PATH_INFO: `/v3/users/b0b7dc063abb4e4fabbed015d9cf73df` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:27.109208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b7dc063abb4e4fabbed015d9cf73df)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:27.136368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:27.224413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d001-d70e-4e2a-b15d-53428139fa9b service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:27.252955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:27.256765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:27.261339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:27.265370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] PATH_INFO: `/v3/users/12ed53b2fa1c456e94a0083cff8bf62f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:27.270612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ed53b2fa1c456e94a0083cff8bf62f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:27.278731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:27.614519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b7dc063abb4e4fabbed015d9cf73df'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.651376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b7dc063abb4e4fabbed015d9cf73df'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.654904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b7dc063abb4e4fabbed015d9cf73df'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.656392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b7dc063abb4e4fabbed015d9cf73df'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.730836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-985e8e6b-7910-4a94-a083-0ae1d2bfa21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b7dc063abb4e4fabbed015d9cf73df'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.782388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ed53b2fa1c456e94a0083cff8bf62f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.783841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ed53b2fa1c456e94a0083cff8bf62f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.814525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ed53b2fa1c456e94a0083cff8bf62f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.816954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ed53b2fa1c456e94a0083cff8bf62f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.916248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 860/1703] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:26 2022] DELETE /identity/v3/users/b0b7dc063abb4e4fabbed015d9cf73df => generated 0 bytes in 1139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:27.921541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:27.922048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:27.922512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:27.971568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6232c1a3-6a82-48da-b5c7-6cd07bfd4b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ed53b2fa1c456e94a0083cff8bf62f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:27.981080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:27.984849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:28.065777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 844/1704] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:26 2022] DELETE /identity/v3/users/12ed53b2fa1c456e94a0083cff8bf62f => generated 0 bytes in 1132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:28.067710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:28.068065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:28.078890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:28.138430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:28.139959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:28.215416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 861/1705] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:28.281546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:28.283560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:28.284537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 845/1706] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:28.374400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:28.376390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:28.451065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd3dcd-5cd4-47c1-93f1-7c563a0677ec None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:28.611653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c46e9-5cee-44d1-b8a4-3f0b1f9a9821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:28.612239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c46e9-5cee-44d1-b8a4-3f0b1f9a9821 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:28.612528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c46e9-5cee-44d1-b8a4-3f0b1f9a9821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:28.612810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c46e9-5cee-44d1-b8a4-3f0b1f9a9821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:28.614108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9699d-1170-45a4-9974-5ced766b0963 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:28.629952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c46e9-5cee-44d1-b8a4-3f0b1f9a9821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:28.632486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c46e9-5cee-44d1-b8a4-3f0b1f9a9821 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:28.635771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:28.638700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:28.641169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:28.641712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:28.661452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 862/1707] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:28.664343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8c028-973d-4d39-94a0-5cb6c9a53eda None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:28.664993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8c028-973d-4d39-94a0-5cb6c9a53eda None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:28.665689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8c028-973d-4d39-94a0-5cb6c9a53eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:28.674772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:28.686589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:28.694941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 846/1708] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:28.699615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:28.699615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:28.713502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6e3fe-5d50-44a8-9570-87d835fc40a8 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:28.726616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e6f8c028-973d-4d39-94a0-5cb6c9a53eda None None] MFA Rules not processed for user `5d079dc5e2b142c7a088648f79d1bb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:28.733532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e6f8c028-973d-4d39-94a0-5cb6c9a53eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:28.734281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:28.735207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:28.735611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:28.735898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] PATH_INFO: `/v3/users/faebd8955e2348b8b54456f0c3c9b00c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:28.739784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faebd8955e2348b8b54456f0c3c9b00c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:28.741969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:29.040372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 863/1709] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:31.324208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:31.351713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faebd8955e2348b8b54456f0c3c9b00c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:31.354844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faebd8955e2348b8b54456f0c3c9b00c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:31.373860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a57016-0e7b-48c7-b87b-83e4e9102e7a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:31.374181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a57016-0e7b-48c7-b87b-83e4e9102e7a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:31.374421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a57016-0e7b-48c7-b87b-83e4e9102e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:31.409524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:31.435592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faebd8955e2348b8b54456f0c3c9b00c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:31.437403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faebd8955e2348b8b54456f0c3c9b00c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:31.469683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a9a57016-0e7b-48c7-b87b-83e4e9102e7a None None] MFA Rules not processed for user `fd3eef3ca9da4302a7c2e52acb8155c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:31.471447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a9a57016-0e7b-48c7-b87b-83e4e9102e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:31.545599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faebd8955e2348b8b54456f0c3c9b00c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:31.612422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 847/1710] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:28 2022] DELETE /identity/v3/users/faebd8955e2348b8b54456f0c3c9b00c => generated 0 bytes in 2915 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:31.613168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:31.624191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:31.777416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 864/1711] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:31.777885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c28fd-c30e-4ac5-a4e1-01805c09314f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:31.778222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c28fd-c30e-4ac5-a4e1-01805c09314f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:31.778470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c28fd-c30e-4ac5-a4e1-01805c09314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:31.852352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f43c28fd-c30e-4ac5-a4e1-01805c09314f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:31.855844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f43c28fd-c30e-4ac5-a4e1-01805c09314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:31.962628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5c677-99b3-4c85-a120-cbc7e0deb8d8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:31.976703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:31.977820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:31.984920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:31.984920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] PATH_INFO: `/v3/users/d37777b81a8e49898b56eafbb2cb2f28` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:31.986436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37777b81a8e49898b56eafbb2cb2f28)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:31.988737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:32.128521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 865/1712] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:32.133824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:32.134128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:32.135752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:32.203308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] MFA Rules not processed for user `cf08ecfc3fed44e2bc1ccff8521a4ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:32.205808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:32.255290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37777b81a8e49898b56eafbb2cb2f28'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:32.258428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37777b81a8e49898b56eafbb2cb2f28'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:32.288371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37777b81a8e49898b56eafbb2cb2f28'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:32.289990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37777b81a8e49898b56eafbb2cb2f28'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:32.419279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37777b81a8e49898b56eafbb2cb2f28'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:32.433428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 866/1713] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jun 23 11:10:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:32.435542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:32.456515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:32.485131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1e3a9-6ab0-461d-aac9-27cba037b240 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:32.504243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:32.510436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:32.510436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:32.510436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:32.510436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:32.514464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 848/1714] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:31 2022] DELETE /identity/v3/users/d37777b81a8e49898b56eafbb2cb2f28 => generated 0 bytes in 902 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:32.516870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:32.602834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:32.604475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:32.622732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 867/1715] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:32.623756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:32.625741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:32.645334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dea1-3f31-4388-ac1b-a7d503b5ca2b service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:32.732441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b84a6d-0c1e-4089-bd9c-508696ee21e5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:32.750092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86569418-d2a8-49da-bd1d-0ac5dc91a72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:32.753210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569418-d2a8-49da-bd1d-0ac5dc91a72b service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:32.753210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569418-d2a8-49da-bd1d-0ac5dc91a72b service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:32.753210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569418-d2a8-49da-bd1d-0ac5dc91a72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:32.772187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86569418-d2a8-49da-bd1d-0ac5dc91a72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:32.774024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86569418-d2a8-49da-bd1d-0ac5dc91a72b service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:32.802562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 849/1716] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:32.815711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:32.819097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:32.819259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:32.822129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:32.822424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] PATH_INFO: `/v3/projects/a2a34e80c2fd4e0fb74d08649b7047a1` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:32.823984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:32.827773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:32.849710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a34e80c2fd4e0fb74d08649b7047a1)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:32.857274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:32.910405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:32.912543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:32.942684 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 850/1717] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:32.945919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:32.946318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:32.973625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1e7bb-9ad9-4c08-a7c2-cecb226dd68a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:32.988067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:32.990059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:32.992607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:32.993033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.015543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:33.029696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:33.057881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a34e80c2fd4e0fb74d08649b7047a1', 'request_id': 'req-6d736ffc-fddf-4b21-b2f3-0824330f2767'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.060164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 851/1718] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:33.061931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:33.076646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a34e80c2fd4e0fb74d08649b7047a1', 'request_id': 'req-6d736ffc-fddf-4b21-b2f3-0824330f2767'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.091579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac397857-6dc0-4026-b829-76fba18e9142 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:33.093573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a34e80c2fd4e0fb74d08649b7047a1', 'request_id': 'req-6d736ffc-fddf-4b21-b2f3-0824330f2767'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.210197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:33.211106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.213266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.213266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.216537 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] The token cache is being invalidate because project a2a34e80c2fd4e0fb74d08649b7047a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:33.217000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.219733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.222534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:33.224230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:33.224360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 868/1719] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:32 2022] DELETE /identity/v3/projects/a2a34e80c2fd4e0fb74d08649b7047a1 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:33.228083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:33.231293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:33.239715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 852/1720] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:33.240325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:33.241936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:33.266896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d736ffc-fddf-4b21-b2f3-0824330f2767 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:33.274254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccedb1-f1c6-4edd-9c13-bb5b304b8e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:33.274937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccedb1-f1c6-4edd-9c13-bb5b304b8e0d service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.275440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccedb1-f1c6-4edd-9c13-bb5b304b8e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.276129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccedb1-f1c6-4edd-9c13-bb5b304b8e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.279838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-43ccedb1-f1c6-4edd-9c13-bb5b304b8e0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:33.337671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e562-bdff-4f5b-b037-6db9386b132c service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:33.348929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:33.349917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.350414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.350820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] PATH_INFO: `/v3/projects/24b0df2cbf744712b463758fcf18a5f5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.353442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24b0df2cbf744712b463758fcf18a5f5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:33.356654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:33.419323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccedb1-f1c6-4edd-9c13-bb5b304b8e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:33.420844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccedb1-f1c6-4edd-9c13-bb5b304b8e0d service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:33.432830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 869/1721] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:33.439398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98061-a330-4168-94a8-a512c1ed6c36 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.439398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98061-a330-4168-94a8-a512c1ed6c36 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.439398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a98061-a330-4168-94a8-a512c1ed6c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.457463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-59a98061-a330-4168-94a8-a512c1ed6c36 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:33.458789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-59a98061-a330-4168-94a8-a512c1ed6c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:33.470989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 870/1722] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:33.486637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe7c65-cee3-4b80-a47c-5e3ea6f5c5f1 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.486637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe7c65-cee3-4b80-a47c-5e3ea6f5c5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.487595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe7c65-cee3-4b80-a47c-5e3ea6f5c5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.513176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-84fe7c65-cee3-4b80-a47c-5e3ea6f5c5f1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:33.514672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-84fe7c65-cee3-4b80-a47c-5e3ea6f5c5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:33.542368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 871/1723] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:33.547504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe7c65-cee3-4b80-a47c-5e3ea6f5c5f1 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:33.564954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24b0df2cbf744712b463758fcf18a5f5', 'request_id': 'req-0ab5f34f-1157-4f43-9e3c-857bae460f2d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.571336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe7c65-cee3-4b80-a47c-5e3ea6f5c5f1 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:33.586455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24b0df2cbf744712b463758fcf18a5f5', 'request_id': 'req-0ab5f34f-1157-4f43-9e3c-857bae460f2d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.609144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24b0df2cbf744712b463758fcf18a5f5', 'request_id': 'req-0ab5f34f-1157-4f43-9e3c-857bae460f2d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.723859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:33.725398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.725701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.725986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.727172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:33.727543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:33.750193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:33.776638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 872/1724] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:33.778209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:33.794032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66deb3-026f-4f03-9bbf-07e7b920e727 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:33.806683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:33.806683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.806683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.806683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.807331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:33.807331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:33.832673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:33.840958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] The token cache is being invalidate because project 24b0df2cbf744712b463758fcf18a5f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:33.842105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.843438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:33.846238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 853/1725] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:33 2022] DELETE /identity/v3/projects/24b0df2cbf744712b463758fcf18a5f5 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:33.852236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:33.854167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:33.887677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f34f-1157-4f43-9e3c-857bae460f2d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:33.904157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:33.906627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:33.906932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:33.907244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] PATH_INFO: `/v3/users/c44abfcbae7142cc89847371cd7c0caa` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:33.914894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44abfcbae7142cc89847371cd7c0caa)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:33.925046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:34.201757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 873/1726] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:33 2022] POST /identity/v3/projects => generated 403 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:34.203070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:34.205153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:34.351012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44abfcbae7142cc89847371cd7c0caa'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:34.352778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44abfcbae7142cc89847371cd7c0caa'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:34.368868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44abfcbae7142cc89847371cd7c0caa'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:34.370372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44abfcbae7142cc89847371cd7c0caa'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:34.385917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbaf7f-27ea-4264-91b2-638726f63019 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:34.393371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:34.395077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:34.395077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:34.395077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:34.396029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:34.399985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:34.448243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44abfcbae7142cc89847371cd7c0caa'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:34.491918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 854/1727] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:33 2022] DELETE /identity/v3/users/c44abfcbae7142cc89847371cd7c0caa => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:34.501785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 874/1728] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:34 2022] POST /identity/v3/users => generated 302 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:34.511272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:34.515773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:34.560528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:34.779598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39282cbf-4c91-444f-81c3-c6bb889378be admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:34.792284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:34.793246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:34.793677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:34.794114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:34.794996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:34.796802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:34.819024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 855/1729] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:34.822998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:34.846068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf0754-8049-44bd-ae2a-3c8ac6016d0e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:34.882826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d78b6-7f61-461e-9fc8-18a31bc48f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:34.883620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d78b6-7f61-461e-9fc8-18a31bc48f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:34.883949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d78b6-7f61-461e-9fc8-18a31bc48f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:34.885173 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d78b6-7f61-461e-9fc8-18a31bc48f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:34.886074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d78b6-7f61-461e-9fc8-18a31bc48f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:34.914997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d78b6-7f61-461e-9fc8-18a31bc48f16 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:34.956521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 875/1730] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:34.961091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:34.983607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cdc60-8b55-45f5-bcd8-a9f8343f02ac admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:34.994925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:34.996418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:34.996726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:34.997012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] PATH_INFO: `/v3/projects/74132fd28aa9485f89309310fd9dc81c/users/fb1265ef79d340158eeeff7c6a332eca/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:35.042991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74132fd28aa9485f89309310fd9dc81c, user_id=fb1265ef79d340158eeeff7c6a332eca, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:35.045736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:35.200148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 856/1731] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:34 2022] PUT /identity/v3/projects/74132fd28aa9485f89309310fd9dc81c/users/fb1265ef79d340158eeeff7c6a332eca/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:35.201946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:35.202718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:35.202718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:35.255219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:35.257072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:35.307502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:35.358363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:35.360178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:35.430201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 876/1732] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:35.499033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:35.500562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:35.582559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10fa76-c50a-4a84-9b04-2254c4a5faf0 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:35.614111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:35.620437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:35.622118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:35.622536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] PATH_INFO: `/v3/projects/668531e7995f41c984e0ca9b1f3a0c92` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:35.628058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=668531e7995f41c984e0ca9b1f3a0c92)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:35.630119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:35.680091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bfdf8-cbde-4f33-be07-f69217d82907 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:35.702952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cce6e-45d4-478d-8abf-5e97bbb7560f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:35.702952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cce6e-45d4-478d-8abf-5e97bbb7560f service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:35.702952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cce6e-45d4-478d-8abf-5e97bbb7560f service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:35.702952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cce6e-45d4-478d-8abf-5e97bbb7560f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:35.712748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cce6e-45d4-478d-8abf-5e97bbb7560f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:35.714814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cce6e-45d4-478d-8abf-5e97bbb7560f service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:35.731376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 877/1733] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:35.898575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904337-edea-42e0-a0d7-fb2ecbcad7f9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:35.899212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904337-edea-42e0-a0d7-fb2ecbcad7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:35.899597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904337-edea-42e0-a0d7-fb2ecbcad7f9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:35.911530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 878/1734] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:10:35 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:35.946836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668531e7995f41c984e0ca9b1f3a0c92', 'request_id': 'req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:35.961682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668531e7995f41c984e0ca9b1f3a0c92', 'request_id': 'req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:35.976322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668531e7995f41c984e0ca9b1f3a0c92', 'request_id': 'req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.007129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:36.007596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:36.008034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:36.066351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:36.067651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:36.116885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] The token cache is being invalidate because project 668531e7995f41c984e0ca9b1f3a0c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:36.117185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.121901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.129134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 857/1735] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:35 2022] DELETE /identity/v3/projects/668531e7995f41c984e0ca9b1f3a0c92 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:36.130319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:36.135389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:36.376000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce1930-22d2-4fe6-b1e7-ad89f22a8172 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:36.387680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:36.389281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:36.389768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:36.390240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] PATH_INFO: `/v3/users/48973709b70d4042a26c04917c445ac4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:36.399668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] RBAC: Authorizing `identity:delete_user(user_id=48973709b70d4042a26c04917c445ac4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:36.399668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:36.414001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 879/1736] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:10:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:36.416563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:36.423828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:36.484996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d45614-af42-47e5-bef3-0f1cab9aa628 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:36.497368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:36.500628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:36.500628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:36.500628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] PATH_INFO: `/v3/projects/924a0f8fe1da49eb9bc3ddac4e09419f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:36.516174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924a0f8fe1da49eb9bc3ddac4e09419f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:36.518168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:36.798714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924a0f8fe1da49eb9bc3ddac4e09419f', 'request_id': 'req-d4cdd737-035d-4158-8f89-d7702a8e9674'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.804259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48973709b70d4042a26c04917c445ac4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.805212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48973709b70d4042a26c04917c445ac4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.828138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924a0f8fe1da49eb9bc3ddac4e09419f', 'request_id': 'req-d4cdd737-035d-4158-8f89-d7702a8e9674'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.837051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48973709b70d4042a26c04917c445ac4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.838001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48973709b70d4042a26c04917c445ac4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.840878 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924a0f8fe1da49eb9bc3ddac4e09419f', 'request_id': 'req-d4cdd737-035d-4158-8f89-d7702a8e9674'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.927957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48973709b70d4042a26c04917c445ac4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:36.981818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 858/1737] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:36 2022] DELETE /identity/v3/users/48973709b70d4042a26c04917c445ac4 => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:36.984238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:36.986173 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:37.001055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:37.089784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] The token cache is being invalidate because project 924a0f8fe1da49eb9bc3ddac4e09419f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:37.090585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.099021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.099021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 880/1738] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:36 2022] DELETE /identity/v3/projects/924a0f8fe1da49eb9bc3ddac4e09419f => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:37.099021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:37.100678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:37.185216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0e4ca-c5a8-4923-aa8f-aa01a1d6267b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:37.199394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:37.200809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:37.201097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:37.202148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:37.206169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:37.254571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:37.259008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:37.307662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 859/1739] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:37.309053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:37.310401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:37.320900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdd737-035d-4158-8f89-d7702a8e9674 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:37.516231 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:37.517119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:37.524422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:37.524422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] PATH_INFO: `/v3/projects/492e3fa83b454f1185d99d89bcc1b1d1` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:37.528666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8f97a-e6dd-4049-a67d-fd907aa7c13f service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:37.531015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] RBAC: Authorizing `identity:delete_project(project_id=492e3fa83b454f1185d99d89bcc1b1d1)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:37.535511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:37.542007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:37.542988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:37.543303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:37.543583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] PATH_INFO: `/v3/users/4ffc5a0054cf4ff7990080439d3ed933` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:37.544894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ffc5a0054cf4ff7990080439d3ed933)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:37.547753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:37.898458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ffc5a0054cf4ff7990080439d3ed933'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.901076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ffc5a0054cf4ff7990080439d3ed933'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.914970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '492e3fa83b454f1185d99d89bcc1b1d1', 'request_id': 'req-9b71daa3-11cd-4a7c-bf60-bef397d0072b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.918108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ffc5a0054cf4ff7990080439d3ed933'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.918938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ffc5a0054cf4ff7990080439d3ed933'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.922653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:37.947817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '492e3fa83b454f1185d99d89bcc1b1d1', 'request_id': 'req-9b71daa3-11cd-4a7c-bf60-bef397d0072b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:37.963144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '492e3fa83b454f1185d99d89bcc1b1d1', 'request_id': 'req-9b71daa3-11cd-4a7c-bf60-bef397d0072b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:38.048505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-98740eaa-2376-41d5-8f39-d81049c73a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ffc5a0054cf4ff7990080439d3ed933'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:38.110741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] The token cache is being invalidate because project 492e3fa83b454f1185d99d89bcc1b1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:38.116031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:38.116150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 860/1740] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:37 2022] DELETE /identity/v3/users/4ffc5a0054cf4ff7990080439d3ed933 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:38.116266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9b71daa3-11cd-4a7c-bf60-bef397d0072b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:38.122122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369cfe8-6eba-45c5-aa56-8cd63d162201 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.122419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369cfe8-6eba-45c5-aa56-8cd63d162201 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.122704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369cfe8-6eba-45c5-aa56-8cd63d162201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.128454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 881/1741] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:37 2022] DELETE /identity/v3/projects/492e3fa83b454f1185d99d89bcc1b1d1 => generated 0 bytes in 1022 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:38.169482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b369cfe8-6eba-45c5-aa56-8cd63d162201 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:38.170873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b369cfe8-6eba-45c5-aa56-8cd63d162201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:38.415251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 861/1742] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:38.424892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8fbfe-7630-4a31-b30d-580917f38877 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.430509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8fbfe-7630-4a31-b30d-580917f38877 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.430509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8fbfe-7630-4a31-b30d-580917f38877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.465711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-29a8fbfe-7630-4a31-b30d-580917f38877 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:38.475143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-29a8fbfe-7630-4a31-b30d-580917f38877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:38.475317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20105550-937a-43a1-9719-5d0e9ee269ea None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.475317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20105550-937a-43a1-9719-5d0e9ee269ea None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.475317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20105550-937a-43a1-9719-5d0e9ee269ea None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.475317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 862/1743] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:10:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:38.504463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e893-ae2b-431e-9705-70ebb548a055 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.504970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e893-ae2b-431e-9705-70ebb548a055 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.507864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e893-ae2b-431e-9705-70ebb548a055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.532462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 882/1744] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:38.534234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8fbfe-7630-4a31-b30d-580917f38877 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.558867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8fbfe-7630-4a31-b30d-580917f38877 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.566550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.567607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.568177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.569087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.570221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:38.570564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.572653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.585562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5be0e893-ae2b-431e-9705-70ebb548a055 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:38.588676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5be0e893-ae2b-431e-9705-70ebb548a055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:38.592480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 883/1745] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:38.596904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.642124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5754009-0c55-461d-9f85-ea3b121a4fbc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:38.656665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:38.659079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:38.660373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:38.661025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:38.662804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:38.666633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:38.792143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 884/1746] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:38.796596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:38.899467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 863/1747] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:38.987432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efca55-c076-42b7-86b7-98e6538b73fd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.000089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.003770 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.004555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.004909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.005733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.021479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.138944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 885/1748] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:38 2022] POST /identity/v3/users => generated 291 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:39.143295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0e893-ae2b-431e-9705-70ebb548a055 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.174457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0e893-ae2b-431e-9705-70ebb548a055 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.195938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.198255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.200615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.201429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.202663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.211891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.228377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 864/1749] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:39.232134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.246498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdf434-51b8-406b-802a-e1a6f3aeb4d0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.253835 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.254535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.254828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.255112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.256026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.268611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.296048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 886/1750] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:39.297382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.324633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cd214-0cf8-48bf-b677-63a60919d10b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.349554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bd69f-72ef-425c-a951-6e3edc068357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.352538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd69f-72ef-425c-a951-6e3edc068357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.354154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd69f-72ef-425c-a951-6e3edc068357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.354640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bd69f-72ef-425c-a951-6e3edc068357 admin admin] PATH_INFO: `/v3/projects/60b2a49bed1c407d8c6330c03b6b02bd/users/69c46a87d71e4f0fb34e9baf1f9b3a0d/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.413381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bd69f-72ef-425c-a951-6e3edc068357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b2a49bed1c407d8c6330c03b6b02bd, user_id=69c46a87d71e4f0fb34e9baf1f9b3a0d, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.423820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bd69f-72ef-425c-a951-6e3edc068357 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.457923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.461027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:39.545141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 865/1751] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:39 2022] PUT /identity/v3/projects/60b2a49bed1c407d8c6330c03b6b02bd/users/69c46a87d71e4f0fb34e9baf1f9b3a0d/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:39.557723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.557723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.557723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.644765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75847-1e1e-41c7-8f02-f66532f9aa06 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.649071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] MFA Rules not processed for user `69c46a87d71e4f0fb34e9baf1f9b3a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:39.651222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:39.750105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.750834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.751125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.751610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.765317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.767973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.773954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 866/1752] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:39.775270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.777985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:39.784478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 887/1753] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:39.786326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:39.935372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf9a5c-c326-461a-a78a-c7294a510e82 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.948668 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded0722-c4cb-4896-8990-44923da0a97a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:39.954280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.954938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:39.955361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.955661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.955942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] PATH_INFO: `/v3/projects/8616647d0b184e1ea67c1368c48cf69c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.956451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:39.958133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:39.958240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8616647d0b184e1ea67c1368c48cf69c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.958747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:39.960737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:39.963408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:39.965575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:40.092483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 888/1754] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:39 2022] POST /identity/v3/projects => generated 379 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:40.092483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:40.095352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:40.242577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8616647d0b184e1ea67c1368c48cf69c', 'request_id': 'req-1a365be0-257c-4c2a-a7da-0dc174137c0d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.259059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8616647d0b184e1ea67c1368c48cf69c', 'request_id': 'req-1a365be0-257c-4c2a-a7da-0dc174137c0d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.266882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85677-6c64-4c77-92b3-c8a8576e856d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:40.269023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8616647d0b184e1ea67c1368c48cf69c', 'request_id': 'req-1a365be0-257c-4c2a-a7da-0dc174137c0d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.322816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:40.397737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] The token cache is being invalidate because project 8616647d0b184e1ea67c1368c48cf69c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:40.398198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.398992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1a365be0-257c-4c2a-a7da-0dc174137c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.404541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 867/1755] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:39 2022] DELETE /identity/v3/projects/8616647d0b184e1ea67c1368c48cf69c => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:40.404541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:40.404541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:40.404541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:40.410545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:40.411906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:40.414449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:40.415356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] PATH_INFO: `/v3/projects/7430841f6e7a456ab00d76ab414d8549` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:40.419322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7430841f6e7a456ab00d76ab414d8549)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:40.435349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:40.484159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] MFA Rules not processed for user `fb1265ef79d340158eeeff7c6a332eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:40.488698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:40.609382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 868/1756] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:40.610675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:40.612519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:40.635593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e406e54-cf15-4f68-bd7a-d5b55bed9e78 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:40.645648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e9bc-75ca-44d9-a3aa-daec6788d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:40.646995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e9bc-75ca-44d9-a3aa-daec6788d4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:40.647344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e9bc-75ca-44d9-a3aa-daec6788d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:40.647641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e9bc-75ca-44d9-a3aa-daec6788d4b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:40.649336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e9bc-75ca-44d9-a3aa-daec6788d4b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:40.653214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e9bc-75ca-44d9-a3aa-daec6788d4b4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:40.697538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:40.740180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7430841f6e7a456ab00d76ab414d8549', 'request_id': 'req-4f650bcc-f26d-4811-bf01-be543ebf5708'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.754008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7430841f6e7a456ab00d76ab414d8549', 'request_id': 'req-4f650bcc-f26d-4811-bf01-be543ebf5708'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.768743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7430841f6e7a456ab00d76ab414d8549', 'request_id': 'req-4f650bcc-f26d-4811-bf01-be543ebf5708'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.849418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 869/1757] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:40 2022] POST /identity/v3/users => generated 296 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:40.849418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:40.849418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:40.849418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:40.886858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] The token cache is being invalidate because project 7430841f6e7a456ab00d76ab414d8549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:40.887198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.888613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:40.893392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 889/1758] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:40 2022] DELETE /identity/v3/projects/7430841f6e7a456ab00d76ab414d8549 => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:40.894418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:40.896167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:40.897967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:40.919661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:41.052454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 870/1759] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:41.053454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.057702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:41.064047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f650bcc-f26d-4811-bf01-be543ebf5708 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.076891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5e980-6e6a-47fb-bb2d-ea6c749eb2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.120870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e980-6e6a-47fb-bb2d-ea6c749eb2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.120870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e980-6e6a-47fb-bb2d-ea6c749eb2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.120870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5e980-6e6a-47fb-bb2d-ea6c749eb2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.120870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5e980-6e6a-47fb-bb2d-ea6c749eb2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.120870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5e980-6e6a-47fb-bb2d-ea6c749eb2a0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.121944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff910c-2d1b-41a2-ae59-fd971897636e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.121944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b674a9-1e4a-4794-b81f-b3207f237055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.121944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b674a9-1e4a-4794-b81f-b3207f237055 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.121944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b674a9-1e4a-4794-b81f-b3207f237055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.122354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b674a9-1e4a-4794-b81f-b3207f237055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.122354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b674a9-1e4a-4794-b81f-b3207f237055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.122354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b674a9-1e4a-4794-b81f-b3207f237055 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.127401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 871/1760] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:41.129179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678bedc-9257-4360-848a-bdd6d1c3c349 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.129581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678bedc-9257-4360-848a-bdd6d1c3c349 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.129961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678bedc-9257-4360-848a-bdd6d1c3c349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.154226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9678bedc-9257-4360-848a-bdd6d1c3c349 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:41.155658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9678bedc-9257-4360-848a-bdd6d1c3c349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:41.165242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 872/1761] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:41.166919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488572-ded2-429f-85bf-a0bdc72096cb None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.167355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488572-ded2-429f-85bf-a0bdc72096cb None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.167742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65488572-ded2-429f-85bf-a0bdc72096cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.200465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-65488572-ded2-429f-85bf-a0bdc72096cb None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:41.201730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-65488572-ded2-429f-85bf-a0bdc72096cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:41.256760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 873/1762] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:41.256760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488572-ded2-429f-85bf-a0bdc72096cb None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.256760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65488572-ded2-429f-85bf-a0bdc72096cb None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.256760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.257726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.257726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.257726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] PATH_INFO: `/v3/projects/6cdcc628e3b049b7b70c26c065248e5c/users/f579be6b3a2d4b4f9e83a568edc4f2c6/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.257726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cdcc628e3b049b7b70c26c065248e5c, user_id=f579be6b3a2d4b4f9e83a568edc4f2c6, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.257726 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.586289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 874/1764] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:41 2022] PUT /identity/v3/projects/6cdcc628e3b049b7b70c26c065248e5c/users/f579be6b3a2d4b4f9e83a568edc4f2c6/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:41.586498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 890/1764] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:40 2022] POST /identity/v3/projects => generated 403 bytes in 692 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:41.588014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.588143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017cc88-0c80-48d0-88f5-63e99331fb20 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.588388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017cc88-0c80-48d0-88f5-63e99331fb20 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.588676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017cc88-0c80-48d0-88f5-63e99331fb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.613942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4017cc88-0c80-48d0-88f5-63e99331fb20 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:41.615199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4017cc88-0c80-48d0-88f5-63e99331fb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:41.694717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b5c5-c0a1-4d47-8c89-1d60e5e4dfb5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.709549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 891/1765] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:41.710756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017cc88-0c80-48d0-88f5-63e99331fb20 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.721245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017cc88-0c80-48d0-88f5-63e99331fb20 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.726290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.727060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.727441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.727781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.728596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.730436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.742441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 892/1766] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:41.743661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.755496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2918e52-e976-41fa-b361-194828e9d7bd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.759429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.760602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.760602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.761137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.762108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.764268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.786371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.787958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.788422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.788709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.789470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:41.789814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.791454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.804816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 875/1767] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:41.809204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.816287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53457f1d-617d-4c8b-acba-10f980b70aa8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.821293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.821963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.822250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.822533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.823336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:41.823650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.825257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.839246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 876/1768] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:41.840114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.853364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36918a9-5bf4-40e6-82ff-4492569f0d97 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.859061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.860092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.860532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.860888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] PATH_INFO: `/v3/projects/6cdcc628e3b049b7b70c26c065248e5c/users/f579be6b3a2d4b4f9e83a568edc4f2c6/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.863248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cdcc628e3b049b7b70c26c065248e5c, user_id=f579be6b3a2d4b4f9e83a568edc4f2c6, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.866182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:41.879758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 877/1769] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:41 2022] PUT /identity/v3/projects/6cdcc628e3b049b7b70c26c065248e5c/users/f579be6b3a2d4b4f9e83a568edc4f2c6/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:41.880360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.926782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 893/1770] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:41 2022] POST /identity/v3/users => generated 308 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:41.929102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:41.952241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effe957-5527-45ff-b407-8b252a3f88e6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.985354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b2fbb-e28e-4fa8-871a-6fbe8ee97d36 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:41.988605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a22d4-9399-4feb-8237-6f03be48f729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:41.989467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a22d4-9399-4feb-8237-6f03be48f729 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:41.989998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a22d4-9399-4feb-8237-6f03be48f729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:41.991867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a22d4-9399-4feb-8237-6f03be48f729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:41.992719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a22d4-9399-4feb-8237-6f03be48f729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:41.994882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a22d4-9399-4feb-8237-6f03be48f729 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.081146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.082043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.082700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.083057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.084322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.091527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.119280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 878/1771] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:41 2022] POST /identity/v3/projects => generated 357 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:42.122665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7f441-812b-4812-a5ba-747ab9df006a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.124139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7f441-812b-4812-a5ba-747ab9df006a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.124514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7f441-812b-4812-a5ba-747ab9df006a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.175022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-23e7f441-812b-4812-a5ba-747ab9df006a None None] MFA Rules not processed for user `f579be6b3a2d4b4f9e83a568edc4f2c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:42.175022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-23e7f441-812b-4812-a5ba-747ab9df006a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:42.243642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 894/1772] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:41 2022] POST /identity/v3/projects => generated 379 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:42.244512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.263276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 879/1773] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:42.264350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7f441-812b-4812-a5ba-747ab9df006a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.356605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7f441-812b-4812-a5ba-747ab9df006a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.364016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.365845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.366235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.366632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.367531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.369889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.380588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033b851-e899-439d-8a7c-9418243a1591 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.384400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.385052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.385336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.386162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.386363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.388019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.412887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 895/1774] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:42.413756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.439282 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805268a6-beab-48b2-a35d-2f1155ea689d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.446275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.447488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.447946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.448386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.449375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.451589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.499951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 880/1775] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:42 2022] POST /identity/v3/users => generated 279 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:42.501385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.503216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:42.566277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df8ccb-2287-41b1-8b30-96d750557f0c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.570443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.571353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.571645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.571930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.595896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 896/1776] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:42 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:42.596931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.618128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163639b5-34be-45d5-9ed1-e9e065d598ae admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.635868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.635868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.635868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.635868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] PATH_INFO: `/v3/projects/408b5d12a6644392b7d5e774e4d2adee/users/427224b6959f4a44a650344fd741d0e3/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.688187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:42.692657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.694530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.702591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=408b5d12a6644392b7d5e774e4d2adee, user_id=427224b6959f4a44a650344fd741d0e3, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.705853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.731353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 881/1777] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:42.733391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.776305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22bd37-298d-4a12-8f7d-48a234fd2dd9 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.782700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.784121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.784631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.785282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.788995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.792607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.806514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 897/1778] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:42 2022] PUT /identity/v3/projects/408b5d12a6644392b7d5e774e4d2adee/users/427224b6959f4a44a650344fd741d0e3/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:42.806514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.820403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 882/1779] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:42.821300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.837655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b124d08-9c8b-47cc-93da-08d27e419ffb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.955918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2887f-b226-4914-a0a6-d0a98ab76c9e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:42.963106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.964021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.964399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.964805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.965569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.968069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.974115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:42.975541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:42.975924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:42.976302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:42.977140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:42.979107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:42.986067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 898/1780] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:42.987597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:42.995244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 883/1781] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:42.995244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.009220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:43.015113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a12a01-70cb-468d-a767-a0731a4c54fd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.017527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef35ee-4cf7-4c9b-9ec0-bd3df1c34e74 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.020861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.022005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.022540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.022874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.023853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.024764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.025201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.025633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.026592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.028665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.041613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.042719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 884/1782] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:43.046673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.047672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.060679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a935438-3099-4f9d-812a-0abd85c01518 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.067799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.069723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.070175 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.070837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] PATH_INFO: `/v3/projects/408b5d12a6644392b7d5e774e4d2adee/users/427224b6959f4a44a650344fd741d0e3/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.073706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=408b5d12a6644392b7d5e774e4d2adee, user_id=427224b6959f4a44a650344fd741d0e3, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.079009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 899/1783] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:43.079009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.080514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.112664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b84d06-a4d8-47cd-9b1c-800e2b8f5065 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.118376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 885/1784] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:43 2022] PUT /identity/v3/projects/408b5d12a6644392b7d5e774e4d2adee/users/427224b6959f4a44a650344fd741d0e3/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:43.119473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.216381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9c04e-c1a1-41b6-b5e5-ed78cfe71f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.217344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9c04e-c1a1-41b6-b5e5-ed78cfe71f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.217642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9c04e-c1a1-41b6-b5e5-ed78cfe71f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.217931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9c04e-c1a1-41b6-b5e5-ed78cfe71f4c admin admin] PATH_INFO: `/v3/projects/5e1ea55074d2449ebf612f66aecbbb3b/users/21288536d941460aaad06438ebac4a03/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.223917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306774b0-a799-400f-87b6-3487d330d16b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.229325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f846f4-d066-4fc2-86d3-2af28d5cd0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.230545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f846f4-d066-4fc2-86d3-2af28d5cd0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.230844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f846f4-d066-4fc2-86d3-2af28d5cd0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.231129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f846f4-d066-4fc2-86d3-2af28d5cd0ec admin admin] PATH_INFO: `/v3/projects/104f001638f64024bb131b1f59828ea3/users/ef1599a5f0994d38987e0afc2530acfa/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.237330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9c04e-c1a1-41b6-b5e5-ed78cfe71f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1ea55074d2449ebf612f66aecbbb3b, user_id=21288536d941460aaad06438ebac4a03, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.241283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9c04e-c1a1-41b6-b5e5-ed78cfe71f4c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.249751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f846f4-d066-4fc2-86d3-2af28d5cd0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=104f001638f64024bb131b1f59828ea3, user_id=ef1599a5f0994d38987e0afc2530acfa, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.256713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f846f4-d066-4fc2-86d3-2af28d5cd0ec admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.334710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 900/1785] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:43 2022] PUT /identity/v3/projects/5e1ea55074d2449ebf612f66aecbbb3b/users/21288536d941460aaad06438ebac4a03/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:43.347358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd2c3f-3070-4d4a-9a6e-0b459225c946 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.347648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd2c3f-3070-4d4a-9a6e-0b459225c946 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.348264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd2c3f-3070-4d4a-9a6e-0b459225c946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.377345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-68bd2c3f-3070-4d4a-9a6e-0b459225c946 None None] MFA Rules not processed for user `21288536d941460aaad06438ebac4a03`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:43.378573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-68bd2c3f-3070-4d4a-9a6e-0b459225c946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:43.413255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 886/1786] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:43 2022] PUT /identity/v3/projects/104f001638f64024bb131b1f59828ea3/users/ef1599a5f0994d38987e0afc2530acfa/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:43.424774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7822f-11d6-4929-8d1f-4b43211b1d52 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.425130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7822f-11d6-4929-8d1f-4b43211b1d52 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.425511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7822f-11d6-4929-8d1f-4b43211b1d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.463110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 901/1787] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:43.464294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2cd7822f-11d6-4929-8d1f-4b43211b1d52 None None] MFA Rules not processed for user `ef1599a5f0994d38987e0afc2530acfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:43.465670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2cd7822f-11d6-4929-8d1f-4b43211b1d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:43.467665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd2c3f-3070-4d4a-9a6e-0b459225c946 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.547974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd2c3f-3070-4d4a-9a6e-0b459225c946 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.551799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 887/1788] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:43 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:43.556360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7822f-11d6-4929-8d1f-4b43211b1d52 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.620739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.621431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.621723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.621998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.622701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.626886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.662912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7822f-11d6-4929-8d1f-4b43211b1d52 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.667717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.668387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.668678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.668960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.669666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.671615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.732402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 902/1789] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:43 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:43.732402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.792749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61b701-a000-426f-82c9-b5cdf5610d9b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.797687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.798644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.798928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.799238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] PATH_INFO: `/v3/users/69c46a87d71e4f0fb34e9baf1f9b3a0d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.800457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c46a87d71e4f0fb34e9baf1f9b3a0d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.802392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:43.875888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 888/1790] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:43 2022] POST /identity/v3/projects => generated 357 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:43.876876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:43.941905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b745b-0dff-4cb7-9489-2751b4bb0221 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:43.948301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:43.949494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:43.949494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:43.949664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:43.950366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:43.952389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.054254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c46a87d71e4f0fb34e9baf1f9b3a0d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.060728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 889/1791] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:43 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:44.060728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.072982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c46a87d71e4f0fb34e9baf1f9b3a0d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.075454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c46a87d71e4f0fb34e9baf1f9b3a0d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.076285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c46a87d71e4f0fb34e9baf1f9b3a0d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.097374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ce8a7-7800-4390-8e90-d4425fe413cf admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.102992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c46a87d71e4f0fb34e9baf1f9b3a0d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:44.107415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.108420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.109475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.109475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.109785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.111793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.232043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 890/1792] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:44 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:44.237226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 903/1793] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:43 2022] DELETE /identity/v3/users/69c46a87d71e4f0fb34e9baf1f9b3a0d => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:44.237226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.237226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:44.240470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.243356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:44.258811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fe7a3-8ac5-4727-a91e-cc571f3a3acc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.328416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0734a02-a483-4975-bfd5-b870854c0114 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.333564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.334548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.340334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.340334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.341009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce6dbd-d25c-4715-90d7-b56a6f6a8497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.341009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce6dbd-d25c-4715-90d7-b56a6f6a8497 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.341192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.341265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce6dbd-d25c-4715-90d7-b56a6f6a8497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.341265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce6dbd-d25c-4715-90d7-b56a6f6a8497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.341265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce6dbd-d25c-4715-90d7-b56a6f6a8497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.341458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.341537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce6dbd-d25c-4715-90d7-b56a6f6a8497 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.350314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 891/1794] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:10:44.351529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 904/1795] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:44.354323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.355143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f12f08-5fb5-4f87-b9c2-fd48c88903c0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.355538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f12f08-5fb5-4f87-b9c2-fd48c88903c0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.355833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f12f08-5fb5-4f87-b9c2-fd48c88903c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.366534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac16b8-1d9c-4bd6-baaa-c179aa040697 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.383668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.384719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.385212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.385580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] PATH_INFO: `/v3/projects/2eb1a5c2d1cf4eed8166fb6516f379df/users/4ac713fd630d4d67ac6634552133da1f/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.415983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-62f12f08-5fb5-4f87-b9c2-fd48c88903c0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:44.417254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-62f12f08-5fb5-4f87-b9c2-fd48c88903c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:44.421034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb1a5c2d1cf4eed8166fb6516f379df, user_id=4ac713fd630d4d67ac6634552133da1f, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.425148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.437018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 892/1796] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:44.439211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f12f08-5fb5-4f87-b9c2-fd48c88903c0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.453076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f12f08-5fb5-4f87-b9c2-fd48c88903c0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.460045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.460618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.460907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.461232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] PATH_INFO: `/v3/projects/64a52c647a7249bba5ee3fc7b21938ad/users/b93b0c8693ed46a6bfb2364a1151628e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.479039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a52c647a7249bba5ee3fc7b21938ad, user_id=b93b0c8693ed46a6bfb2364a1151628e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.481794 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.533350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 905/1797] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:44 2022] PUT /identity/v3/projects/2eb1a5c2d1cf4eed8166fb6516f379df/users/4ac713fd630d4d67ac6634552133da1f/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:44.534433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.536500 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:44.630073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47ab9f-1c14-475a-b395-2bb18b8ab9eb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.635475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.636964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.636964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.636964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.682912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 893/1798] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:44 2022] PUT /identity/v3/projects/64a52c647a7249bba5ee3fc7b21938ad/users/b93b0c8693ed46a6bfb2364a1151628e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:44.686412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.712705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.714371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.728030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 906/1799] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:44.729295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.739945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e768e0f-1499-459c-8e03-3865c798d82f service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.810218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.810906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.811269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.811564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.812316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.814005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.827137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 907/1800] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:44.831629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:44.844016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79bdba-dda9-41bf-bf21-e1656e660d36 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.851960 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a2f93-ae1a-456c-8485-26b9f15f4ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:44.853468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a2f93-ae1a-456c-8485-26b9f15f4ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.853949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a2f93-ae1a-456c-8485-26b9f15f4ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.854430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a2f93-ae1a-456c-8485-26b9f15f4ac2 admin admin] PATH_INFO: `/v3/projects/64a52c647a7249bba5ee3fc7b21938ad/users/b93b0c8693ed46a6bfb2364a1151628e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.858068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a2f93-ae1a-456c-8485-26b9f15f4ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a52c647a7249bba5ee3fc7b21938ad, user_id=b93b0c8693ed46a6bfb2364a1151628e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:44.861659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a2f93-ae1a-456c-8485-26b9f15f4ac2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:44.884631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 908/1801] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:44 2022] PUT /identity/v3/projects/64a52c647a7249bba5ee3fc7b21938ad/users/b93b0c8693ed46a6bfb2364a1151628e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:44.886210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:44.886509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:44.886794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:44.917434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506f83-7ea0-4d74-8ee1-f34dda6e9a41 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:44.917676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] MFA Rules not processed for user `b93b0c8693ed46a6bfb2364a1151628e`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:44.919118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:45.034885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 909/1802] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:45.051330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.052555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.053438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.053906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.056374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.057610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:45.060032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:45.060160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.085953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 894/1803] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:45.090884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:45.128106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a6a29-6b75-4aa1-8cba-f8dc3658d288 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.149992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035879af-fef5-4b13-a54e-74f561c55627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.154990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035879af-fef5-4b13-a54e-74f561c55627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.156934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035879af-fef5-4b13-a54e-74f561c55627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.157400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035879af-fef5-4b13-a54e-74f561c55627 admin admin] PATH_INFO: `/v3/projects/2eb1a5c2d1cf4eed8166fb6516f379df/users/4ac713fd630d4d67ac6634552133da1f/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.163810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035879af-fef5-4b13-a54e-74f561c55627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb1a5c2d1cf4eed8166fb6516f379df, user_id=4ac713fd630d4d67ac6634552133da1f, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.171795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035879af-fef5-4b13-a54e-74f561c55627 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.195200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 895/1804] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:45 2022] PUT /identity/v3/projects/2eb1a5c2d1cf4eed8166fb6516f379df/users/4ac713fd630d4d67ac6634552133da1f/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:45.207026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.207948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.208256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.229082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1fc6e-cb98-4cbf-9372-6c6c565aee28 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.251608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] MFA Rules not processed for user `4ac713fd630d4d67ac6634552133da1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:45.253949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:45.356173 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.358753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.359053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.359786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.416165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 896/1805] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:45 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:45.425119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:45.426556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:45.441156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e126c-7277-4141-8438-e66c6d79ab55 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.446441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87c06e-22e9-4b28-a92e-11edcd9bba8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:45.447342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87c06e-22e9-4b28-a92e-11edcd9bba8c service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.447674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87c06e-22e9-4b28-a92e-11edcd9bba8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.447959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87c06e-22e9-4b28-a92e-11edcd9bba8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.464310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87c06e-22e9-4b28-a92e-11edcd9bba8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.466883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87c06e-22e9-4b28-a92e-11edcd9bba8c service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.475577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:45.477910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:45.494507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 897/1806] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:45 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:45.503908 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 910/1807] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:45.949020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683299f-e0cc-4246-bcc0-8c9443cef9fc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:45.950809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683299f-e0cc-4246-bcc0-8c9443cef9fc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:45.953148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:45.953498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683299f-e0cc-4246-bcc0-8c9443cef9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:45.966170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f655dc4-9459-4813-9c10-ea32ca7ed1a5 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:45.997764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.000731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.002124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.002673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.007221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.111139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3683299f-e0cc-4246-bcc0-8c9443cef9fc None None] MFA Rules not processed for user `427224b6959f4a44a650344fd741d0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:46.116631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3683299f-e0cc-4246-bcc0-8c9443cef9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.155034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.158940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.167515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 911/1808] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:45 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:46.227380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:46.229634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:46.306334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 898/1809] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:46.325985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e863c-f8f5-4cea-a4bd-523c9f310ae6 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:46.331210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:46.332338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:46.332625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:46.332902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] PATH_INFO: `/v3/projects/60b2a49bed1c407d8c6330c03b6b02bd` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:46.334652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b2a49bed1c407d8c6330c03b6b02bd)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:46.336661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:46.380571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:46.662315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b2a49bed1c407d8c6330c03b6b02bd', 'request_id': 'req-c1de6710-0685-418f-ad6b-8ee121848fc4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.672146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b2a49bed1c407d8c6330c03b6b02bd', 'request_id': 'req-c1de6710-0685-418f-ad6b-8ee121848fc4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:46.679940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b2a49bed1c407d8c6330c03b6b02bd', 'request_id': 'req-c1de6710-0685-418f-ad6b-8ee121848fc4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.765213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] The token cache is being invalidate because project 60b2a49bed1c407d8c6330c03b6b02bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:47.765899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.767658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c1de6710-0685-418f-ad6b-8ee121848fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:47.769580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 912/1810] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:46 2022] DELETE /identity/v3/projects/60b2a49bed1c407d8c6330c03b6b02bd => generated 0 bytes in 1543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:47.774074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683299f-e0cc-4246-bcc0-8c9443cef9fc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:47.786177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683299f-e0cc-4246-bcc0-8c9443cef9fc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:47.858851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:47.859955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:47.860460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:47.860851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] PATH_INFO: `/v3/users/f579be6b3a2d4b4f9e83a568edc4f2c6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:47.862289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f579be6b3a2d4b4f9e83a568edc4f2c6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:47.864637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:47.916193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:48.196166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f579be6b3a2d4b4f9e83a568edc4f2c6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.197054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f579be6b3a2d4b4f9e83a568edc4f2c6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.208246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f579be6b3a2d4b4f9e83a568edc4f2c6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.208246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f579be6b3a2d4b4f9e83a568edc4f2c6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.286527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01295ac2-99b1-4ab4-a2e7-3444e4f23da9 None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.286972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01295ac2-99b1-4ab4-a2e7-3444e4f23da9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.287470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01295ac2-99b1-4ab4-a2e7-3444e4f23da9 None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.289281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 913/1811] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:10:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:48.309916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.309916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.309916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.322245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f579be6b3a2d4b4f9e83a568edc4f2c6'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:48.355831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 899/1812] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:47 2022] DELETE /identity/v3/users/f579be6b3a2d4b4f9e83a568edc4f2c6 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:48.356083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:48.356083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.416975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.418594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.453592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 914/1813] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:10:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:48.465183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.467535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.483039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38434910-ddf3-410c-83c3-8ea5cadbb48d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.489704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77657285-7b48-44cd-b6d3-81675b0f7928 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.492323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.493074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.493368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.493652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.495568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93b0c8693ed46a6bfb2364a1151628e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a52c647a7249bba5ee3fc7b21938ad', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.496308 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.496667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.497029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.497959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:48.498344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.500118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.502694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.504301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.513078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 900/1814] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:10:48.520205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.520527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 915/1815] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:48.532625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482062b-a44c-475a-a20b-1fec9db14faf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.539763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93b0c8693ed46a6bfb2364a1151628e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a52c647a7249bba5ee3fc7b21938ad', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.540427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.540724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.541008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.541743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.544021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.679949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 901/1816] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:48 2022] POST /identity/v3/projects => generated 355 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:48.686522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.716955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420eda5-32f0-4bf7-af89-ab20e194b402 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.788987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93b0c8693ed46a6bfb2364a1151628e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a52c647a7249bba5ee3fc7b21938ad', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.789619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.789918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.790208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.790952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:48.791307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.792926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.805981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 916/1817] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:48.849343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.861362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d749d-0197-462c-aad9-3b29d415a9e2 tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.866901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b93b0c8693ed46a6bfb2364a1151628e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a52c647a7249bba5ee3fc7b21938ad', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.868011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.868370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.868717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] PATH_INFO: `/v3/projects/6658624fc46d47e2b31448d10666392c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.870169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6658624fc46d47e2b31448d10666392c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:48.872224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:48.901809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:48.903755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:48.983597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d263c-b7a1-40cb-8745-6bc168830cbf tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:48.993405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:48.996468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:48.996777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:48.997066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] PATH_INFO: `/v3/projects/6cdcc628e3b049b7b70c26c065248e5c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:48.999082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cdcc628e3b049b7b70c26c065248e5c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.001060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.008340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6658624fc46d47e2b31448d10666392c', 'request_id': 'req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.021144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6658624fc46d47e2b31448d10666392c', 'request_id': 'req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.029855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6658624fc46d47e2b31448d10666392c', 'request_id': 'req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.146080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] The token cache is being invalidate because project 6658624fc46d47e2b31448d10666392c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:49.146380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.147160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e36dbf29-2c05-4f32-b3fd-8c33a06040cd tempest-VolumeQuotaClassesTest-2103080950 tempest-VolumeQuotaClassesTest-2103080950-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.148767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 902/1818] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:48 2022] DELETE /identity/v3/projects/6658624fc46d47e2b31448d10666392c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:49.156641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767178f-b8bb-4325-bc59-7c09b386c53a None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.156961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767178f-b8bb-4325-bc59-7c09b386c53a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.157252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767178f-b8bb-4325-bc59-7c09b386c53a None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.161620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 903/1819] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:10:49 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:10:49.183863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cdcc628e3b049b7b70c26c065248e5c', 'request_id': 'req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.192852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd353f2-2459-45f2-bdbe-3feec7a2c415 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.193462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd353f2-2459-45f2-bdbe-3feec7a2c415 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.193864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd353f2-2459-45f2-bdbe-3feec7a2c415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.203449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cdcc628e3b049b7b70c26c065248e5c', 'request_id': 'req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.215531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cdcc628e3b049b7b70c26c065248e5c', 'request_id': 'req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.226052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4dd353f2-2459-45f2-bdbe-3feec7a2c415 None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:49.227619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4dd353f2-2459-45f2-bdbe-3feec7a2c415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.322375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 904/1820] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:49.352564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] The token cache is being invalidate because project 6cdcc628e3b049b7b70c26c065248e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:49.352989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.354341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.360422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 917/1821] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:48 2022] DELETE /identity/v3/projects/6cdcc628e3b049b7b70c26c065248e5c => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:49.361859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.362051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.363095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.390051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:49.391468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.414726 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.416710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.440254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa4e50-806f-4c23-99d9-4c0f2f584d9e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.451316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054a72f-873a-43f7-a366-9ae99b585424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.452514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054a72f-873a-43f7-a366-9ae99b585424 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.455766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054a72f-873a-43f7-a366-9ae99b585424 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.455766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054a72f-873a-43f7-a366-9ae99b585424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.469848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2054a72f-873a-43f7-a366-9ae99b585424 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.488767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054a72f-873a-43f7-a366-9ae99b585424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.493514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054a72f-873a-43f7-a366-9ae99b585424 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.512447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 905/1822] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:49.517161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 918/1823] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:49.520525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.522730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.544879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e65e5-2c01-47ce-9a47-dc36cbc0bbf9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.548396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a162-eefa-4db5-bc96-fb39118d3c03 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.550958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a162-eefa-4db5-bc96-fb39118d3c03 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.556906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332a162-eefa-4db5-bc96-fb39118d3c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.557171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.557171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.557171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.557171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] PATH_INFO: `/v3/users/fb1265ef79d340158eeeff7c6a332eca` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.557519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1265ef79d340158eeeff7c6a332eca)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.562802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.608249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9332a162-eefa-4db5-bc96-fb39118d3c03 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:49.609561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9332a162-eefa-4db5-bc96-fb39118d3c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.627456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 919/1824] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:49.632400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332a162-eefa-4db5-bc96-fb39118d3c03 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.664283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332a162-eefa-4db5-bc96-fb39118d3c03 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.673206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.673824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.674114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.674397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.676172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:49.676172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.677301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.691290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 920/1825] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:10:49.696125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.706969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce6c6a-3293-4e38-ac5f-351551b4ac6b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.712063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.712746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.713058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.713559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.714228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.716348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.795096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1265ef79d340158eeeff7c6a332eca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.795994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1265ef79d340158eeeff7c6a332eca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.815592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1265ef79d340158eeeff7c6a332eca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.817893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1265ef79d340158eeeff7c6a332eca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:49.827312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 921/1826] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:49 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:49.828729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:49.830681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:49.907435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76f75a-3b93-4f12-99f9-bf1804f89499 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:49.913068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:49.913857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:49.914088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:49.914371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:49.915080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:49.917066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:49.991661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1265ef79d340158eeeff7c6a332eca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.010195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 906/1827] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:49 2022] DELETE /identity/v3/users/fb1265ef79d340158eeeff7c6a332eca => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:50.048163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 922/1828] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:49 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:50.052544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.086396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.089906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:50.111953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad84b8-03e6-476a-bc89-e410111d8f60 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:50.179672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b179f64-3e3b-4a25-8ae3-febe7570799d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:50.185867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:50.186549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.186833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.187116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.187388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:50.187905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:50.188413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.189174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.189174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] PATH_INFO: `/v3/users/21288536d941460aaad06438ebac4a03` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.189602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:50.190499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] RBAC: Authorizing `identity:delete_user(user_id=21288536d941460aaad06438ebac4a03)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:50.192545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:50.205545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 907/1829] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:50.206764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.217968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ad763-84ff-4e9a-b8f1-8809d2356142 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:50.222680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:50.223487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.228520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.228520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.228520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:50.228520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:50.236744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 908/1830] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:50.241469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.251637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca037246-8b5e-4d89-a8cd-5d9adcaa7255 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:50.256832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:50.257800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.258093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.258374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] PATH_INFO: `/v3/projects/c9b195ac106e40ebb3067c3b6c5f4d8d/users/7f2a7fb5016841c68af106e5c860a364/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.273875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b195ac106e40ebb3067c3b6c5f4d8d, user_id=7f2a7fb5016841c68af106e5c860a364, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:50.281194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:50.416427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 909/1831] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:50 2022] PUT /identity/v3/projects/c9b195ac106e40ebb3067c3b6c5f4d8d/users/7f2a7fb5016841c68af106e5c860a364/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:50.417286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.419542 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:50.434149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21288536d941460aaad06438ebac4a03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.445037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21288536d941460aaad06438ebac4a03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.445880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21288536d941460aaad06438ebac4a03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.447062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21288536d941460aaad06438ebac4a03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.463007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-593f61ff-a163-40c4-abd8-325dc686619c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21288536d941460aaad06438ebac4a03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.512104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e39d1-f0da-4491-a45c-ed739df61822 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:50.517222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:50.518212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.518603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.518987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] PATH_INFO: `/v3/projects/74132fd28aa9485f89309310fd9dc81c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.520712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74132fd28aa9485f89309310fd9dc81c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:50.522538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:50.541192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 923/1832] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:50 2022] DELETE /identity/v3/users/21288536d941460aaad06438ebac4a03 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:50.542304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092ae9-9574-4a33-ad93-1f51edb4b585 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.542593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092ae9-9574-4a33-ad93-1f51edb4b585 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.542871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092ae9-9574-4a33-ad93-1f51edb4b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.571861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fb092ae9-9574-4a33-ad93-1f51edb4b585 None None] MFA Rules not processed for user `7f2a7fb5016841c68af106e5c860a364`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:50.573201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fb092ae9-9574-4a33-ad93-1f51edb4b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:50.656177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 924/1833] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:50.657826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.658208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.658542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.686625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74132fd28aa9485f89309310fd9dc81c', 'request_id': 'req-3722afdb-19ce-44ae-9579-e5a433d7bd59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.691436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74132fd28aa9485f89309310fd9dc81c', 'request_id': 'req-3722afdb-19ce-44ae-9579-e5a433d7bd59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.699837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74132fd28aa9485f89309310fd9dc81c', 'request_id': 'req-3722afdb-19ce-44ae-9579-e5a433d7bd59'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.700823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:50.702087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:50.785360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] The token cache is being invalidate because project 74132fd28aa9485f89309310fd9dc81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:50.786065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.786868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:50.787257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 925/1834] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:50.788719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.790381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:50.795683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 910/1835] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:50 2022] DELETE /identity/v3/projects/74132fd28aa9485f89309310fd9dc81c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:50.797904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.802013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:50.808063 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957adc51-d8d3-47fe-bec8-587b3494bcbb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:50.814734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:50.815626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.815914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.816203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.817526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:50.819904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:50.828634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722afdb-19ce-44ae-9579-e5a433d7bd59 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:50.844666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:50.845706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:50.846002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:50.846288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] PATH_INFO: `/v3/users/427224b6959f4a44a650344fd741d0e3` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:50.848553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=427224b6959f4a44a650344fd741d0e3)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:50.852083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:50.950069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 926/1836] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:50 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:50.950069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:50.950835 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:51.023983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1cea3-f0fc-4948-9115-aeb3cf8239d4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:51.028247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:51.029155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:51.029676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:51.030172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:51.032355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:51.105899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '427224b6959f4a44a650344fd741d0e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:51.107267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '427224b6959f4a44a650344fd741d0e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:51.112758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:51.114220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:51.116428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '427224b6959f4a44a650344fd741d0e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:51.117272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '427224b6959f4a44a650344fd741d0e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:51.124346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 927/1837] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:51.125292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:51.126608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:51.138727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437a040-c872-4c3b-b7d4-d99d175975ab service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:51.143546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:51.144228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:51.144508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:51.144789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:51.145653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:51.148255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:51.288102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '427224b6959f4a44a650344fd741d0e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:51.307121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 911/1838] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:50 2022] DELETE /identity/v3/users/427224b6959f4a44a650344fd741d0e3 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:51.323129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:51.331227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 928/1839] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:51 2022] POST /identity/v3/users => generated 298 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:51.336216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:51.338122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:51.430627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ac553-4e8a-4204-9fab-0accb284bc23 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:51.448553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:51.450524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:51.454112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:51.454112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:51.454112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:51.456550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:51.472673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 912/1840] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:51.474168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:51.488207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edf46a-e3da-41a7-9e72-dd4553bdbd17 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:51.495663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:51.496625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:51.496915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:51.497196 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] PATH_INFO: `/v3/projects/de587413ae4e42e3a2ed796ab9c2cf9e/users/d1a70ded44cb4218b5d73d3451e9e5e3/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:51.517450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de587413ae4e42e3a2ed796ab9c2cf9e, user_id=d1a70ded44cb4218b5d73d3451e9e5e3, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:51.520830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:51.603575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 929/1841] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:51 2022] PUT /identity/v3/projects/de587413ae4e42e3a2ed796ab9c2cf9e/users/d1a70ded44cb4218b5d73d3451e9e5e3/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:51.607526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:51.617631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0646eb-84b6-4f77-a986-8e13f398e3a7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:51.627671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:51.641398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:51.648171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:51.743532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:51.748401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:51.748813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:51.749199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:51.750204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:51.752397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:51.783112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 913/1842] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:51.784390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:51.799306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:51.801679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc7ce-66b1-4fab-8399-0c269eb4b5b1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:51.809456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:51.810465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:51.811213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:51.811566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] PATH_INFO: `/v3/projects/5e1ea55074d2449ebf612f66aecbbb3b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:51.812894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1ea55074d2449ebf612f66aecbbb3b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:51.816058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:51.820297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac8c0a-74ba-4ffa-a65a-28086ba456de admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:51.825359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:51.826252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:51.826534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:51.826820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] PATH_INFO: `/v3/projects/408b5d12a6644392b7d5e774e4d2adee` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:51.828138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=408b5d12a6644392b7d5e774e4d2adee)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:51.830281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:52.000838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1ea55074d2449ebf612f66aecbbb3b', 'request_id': 'req-f92920a7-6a89-4a33-b12b-412e6180957c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.007451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1ea55074d2449ebf612f66aecbbb3b', 'request_id': 'req-f92920a7-6a89-4a33-b12b-412e6180957c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.014483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1ea55074d2449ebf612f66aecbbb3b', 'request_id': 'req-f92920a7-6a89-4a33-b12b-412e6180957c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.029763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '408b5d12a6644392b7d5e774e4d2adee', 'request_id': 'req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.036259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '408b5d12a6644392b7d5e774e4d2adee', 'request_id': 'req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.046196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '408b5d12a6644392b7d5e774e4d2adee', 'request_id': 'req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.109952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] The token cache is being invalidate because project 5e1ea55074d2449ebf612f66aecbbb3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:52.110355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.111146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.119305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 930/1843] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:51 2022] DELETE /identity/v3/projects/5e1ea55074d2449ebf612f66aecbbb3b => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:52.119305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:52.119305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.184466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92920a7-6a89-4a33-b12b-412e6180957c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:52.190261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:52.191471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.191876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.192162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] PATH_INFO: `/v3/projects/de587413ae4e42e3a2ed796ab9c2cf9e/users/d1a70ded44cb4218b5d73d3451e9e5e3/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.194476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de587413ae4e42e3a2ed796ab9c2cf9e, user_id=d1a70ded44cb4218b5d73d3451e9e5e3, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:52.197753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:52.226121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 931/1844] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:52 2022] PUT /identity/v3/projects/de587413ae4e42e3a2ed796ab9c2cf9e/users/d1a70ded44cb4218b5d73d3451e9e5e3/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:52.226121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:52.226121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.240282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] The token cache is being invalidate because project 408b5d12a6644392b7d5e774e4d2adee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:52.240915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.241469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-edda0865-6dcf-41ff-a46b-35f4bf3e8cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.244058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 914/1845] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:51 2022] DELETE /identity/v3/projects/408b5d12a6644392b7d5e774e4d2adee => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:52.247971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556f888-feba-4150-a762-736c045605e4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.248463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556f888-feba-4150-a762-736c045605e4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.248885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556f888-feba-4150-a762-736c045605e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.284110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a556f888-feba-4150-a762-736c045605e4 None None] MFA Rules not processed for user `d1a70ded44cb4218b5d73d3451e9e5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:52.284110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a556f888-feba-4150-a762-736c045605e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.305176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bb3e2-c692-4a2c-8406-09892947676f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:52.309946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:52.310824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.311109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.311421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] PATH_INFO: `/v3/users/b93b0c8693ed46a6bfb2364a1151628e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.312620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93b0c8693ed46a6bfb2364a1151628e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:52.314549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:52.348950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 915/1846] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:52.351159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.351542 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.351890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.373858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:52.373858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.381823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 916/1847] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:52.382704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:52.384036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.448657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b888f6-2603-42e9-a0e0-2bf9810362c1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:52.453170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d3570-9a5a-46f3-9a9a-a5b3105a3ece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:52.453829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3570-9a5a-46f3-9a9a-a5b3105a3ece service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.454119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3570-9a5a-46f3-9a9a-a5b3105a3ece service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.454401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3570-9a5a-46f3-9a9a-a5b3105a3ece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.456289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-350d3570-9a5a-46f3-9a9a-a5b3105a3ece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.527471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d3570-9a5a-46f3-9a9a-a5b3105a3ece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:52.528998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d3570-9a5a-46f3-9a9a-a5b3105a3ece service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:52.540490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 917/1848] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:52.541953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ec97-d683-4835-bec9-7ddc9038eb4c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.542250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ec97-d683-4835-bec9-7ddc9038eb4c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.542635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ec97-d683-4835-bec9-7ddc9038eb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.563533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9654ec97-d683-4835-bec9-7ddc9038eb4c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:52.564759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9654ec97-d683-4835-bec9-7ddc9038eb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.572010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93b0c8693ed46a6bfb2364a1151628e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.574888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 918/1849] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:52.579434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654ec97-d683-4835-bec9-7ddc9038eb4c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:52.588591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93b0c8693ed46a6bfb2364a1151628e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.589719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93b0c8693ed46a6bfb2364a1151628e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.590404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93b0c8693ed46a6bfb2364a1151628e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.591412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654ec97-d683-4835-bec9-7ddc9038eb4c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:52.597115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:52.598206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.598206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.598568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.599417 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:52.599872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:52.601788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:52.608262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93b0c8693ed46a6bfb2364a1151628e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:52.618022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 919/1850] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:10:52.624411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:52.626913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:52.640395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9350c2-202d-4aa2-b228-1baa3617829e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:52.645705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:52.647220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.647515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.647796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.648488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:52.650532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:52.723565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 932/1851] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:10:52 2022] DELETE /identity/v3/users/b93b0c8693ed46a6bfb2364a1151628e => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:52.764995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 920/1852] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:52 2022] POST /identity/v3/projects => generated 401 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:52.766704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:52.859330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c6bfa-df13-411b-a30d-753e86e436c6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:52.929056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:52.929746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:52.930035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:52.930316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:52.931027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:52.938028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.111806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 933/1853] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:52 2022] POST /identity/v3/users => generated 301 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:53.116792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.131712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1bcbf-cb6b-48a3-a3a4-d945cb9f8725 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.144272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.146206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.147260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.148125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.159911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.168099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.181326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 921/1854] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:53.185124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.196509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1ef5d-6f22-4fdd-bf55-5be5681c3567 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.202588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.203339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.203810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.204099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.204844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.207325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.221517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 934/1855] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:53.225584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.241280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4445c2-6371-4754-9744-735608d07120 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.250865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61676b28-0e2a-4b00-b119-a69ec344a236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.251881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61676b28-0e2a-4b00-b119-a69ec344a236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.252348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61676b28-0e2a-4b00-b119-a69ec344a236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.252861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61676b28-0e2a-4b00-b119-a69ec344a236 admin admin] PATH_INFO: `/v3/projects/986a20135f124bc9a34129de3d7f280e/users/d41b3a4426ae43138186d334085e5564/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.274887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61676b28-0e2a-4b00-b119-a69ec344a236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=986a20135f124bc9a34129de3d7f280e, user_id=d41b3a4426ae43138186d334085e5564, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.277733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61676b28-0e2a-4b00-b119-a69ec344a236 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.480801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.480979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 922/1856] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:53 2022] PUT /identity/v3/projects/986a20135f124bc9a34129de3d7f280e/users/d41b3a4426ae43138186d334085e5564/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:53.491973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6fe41-715c-42ef-9d0a-e62621a216a6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.492474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6fe41-715c-42ef-9d0a-e62621a216a6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.492909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6fe41-715c-42ef-9d0a-e62621a216a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.496150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:53.518468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9c56-569f-4949-9399-4a17f8a8f0d1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.534887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4af6fe41-715c-42ef-9d0a-e62621a216a6 None None] MFA Rules not processed for user `d41b3a4426ae43138186d334085e5564`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:53.536620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4af6fe41-715c-42ef-9d0a-e62621a216a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:53.581384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.582259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.582916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.582916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] PATH_INFO: `/v3/projects/64a52c647a7249bba5ee3fc7b21938ad` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.584207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a52c647a7249bba5ee3fc7b21938ad)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.586803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.604143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 923/1857] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:53.608659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6fe41-715c-42ef-9d0a-e62621a216a6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.621560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6fe41-715c-42ef-9d0a-e62621a216a6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.631940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.632624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.632917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.633198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.633898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.636236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:53.803340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a52c647a7249bba5ee3fc7b21938ad', 'request_id': 'req-66d13e5c-7877-477e-a0b2-b30f0b9de455'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:53.808923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a52c647a7249bba5ee3fc7b21938ad', 'request_id': 'req-66d13e5c-7877-477e-a0b2-b30f0b9de455'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:53.814936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a52c647a7249bba5ee3fc7b21938ad', 'request_id': 'req-66d13e5c-7877-477e-a0b2-b30f0b9de455'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:53.858475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 924/1858] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:53 2022] POST /identity/v3/projects => generated 401 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:53.859529 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:53.934175 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a78d4-7e5b-4655-a16b-fdac7f3d71e7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:53.938838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:53.939553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:53.939845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:53.940127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:53.940822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:53.942757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.004499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] The token cache is being invalidate because project 64a52c647a7249bba5ee3fc7b21938ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:10:54.004811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:54.005449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-66d13e5c-7877-477e-a0b2-b30f0b9de455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:10:54.007845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 935/1859] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:10:53 2022] DELETE /identity/v3/projects/64a52c647a7249bba5ee3fc7b21938ad => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:54.013409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25456-a863-4a05-a55d-46b5c3caa7d5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.013699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25456-a863-4a05-a55d-46b5c3caa7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.013983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25456-a863-4a05-a55d-46b5c3caa7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.035076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7dd25456-a863-4a05-a55d-46b5c3caa7d5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:54.036739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7dd25456-a863-4a05-a55d-46b5c3caa7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:54.045145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 936/1860] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:54.052532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90371e23-dd8a-4685-a32e-9276aa7c1ca9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.052864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90371e23-dd8a-4685-a32e-9276aa7c1ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.053149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90371e23-dd8a-4685-a32e-9276aa7c1ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.072640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-90371e23-dd8a-4685-a32e-9276aa7c1ca9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:54.073840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-90371e23-dd8a-4685-a32e-9276aa7c1ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:54.081642 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 937/1861] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:10:54.086248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90371e23-dd8a-4685-a32e-9276aa7c1ca9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.095052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90371e23-dd8a-4685-a32e-9276aa7c1ca9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.099901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.100551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.100837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.101118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.101896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:10:54.102205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.103847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.115095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 938/1862] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:10:54.119106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.128311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b601ba2-f98f-4c0d-99e4-f68f6c7c05b9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.133698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.134339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.134623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.134900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.135607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.137603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.189700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 925/1863] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:53 2022] POST /identity/v3/users => generated 307 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:54.190713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.192810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:54.206837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fd14f-1448-40df-bee8-e2de0452d937 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.212968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.213720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.214072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.214414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.215353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.216955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.227724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 926/1864] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:54.236445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.246025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6e14e-69e7-46d1-8486-f08db1e5c706 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.250546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.251407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.251711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.251987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] PATH_INFO: `/v3/projects/8c332ceb21f3474b89bbcbe58020ac3b/users/61d9d17995cc4190b448625b13d82c7b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.266847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c332ceb21f3474b89bbcbe58020ac3b, user_id=61d9d17995cc4190b448625b13d82c7b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.269523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.318102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 939/1865] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:10:54 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:10:54.323057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.398149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752abb6d-6abb-4fe3-9abe-c1f1f87d5d4e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.405283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.406035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.406415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.406862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.408164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.411072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.421886 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 927/1866] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:54 2022] PUT /identity/v3/projects/8c332ceb21f3474b89bbcbe58020ac3b/users/61d9d17995cc4190b448625b13d82c7b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:10:54.427563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.502588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a119f-3cae-4ad4-b018-558ed1a154dd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.507813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.508509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.508970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.509258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.510002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.511634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.522722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 928/1867] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:10:54.526523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.540547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad8d0b-04b1-41ea-b361-2d2147fb2da0 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.555681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687fb14-d25a-43be-bdf5-50f9eef90d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.556796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fb14-d25a-43be-bdf5-50f9eef90d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.557097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fb14-d25a-43be-bdf5-50f9eef90d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.557387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fb14-d25a-43be-bdf5-50f9eef90d11 admin admin] PATH_INFO: `/v3/projects/8c332ceb21f3474b89bbcbe58020ac3b/users/61d9d17995cc4190b448625b13d82c7b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.560384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687fb14-d25a-43be-bdf5-50f9eef90d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c332ceb21f3474b89bbcbe58020ac3b, user_id=61d9d17995cc4190b448625b13d82c7b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.563282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687fb14-d25a-43be-bdf5-50f9eef90d11 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.576801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 929/1868] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:54 2022] PUT /identity/v3/projects/8c332ceb21f3474b89bbcbe58020ac3b/users/61d9d17995cc4190b448625b13d82c7b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:54.588868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.589156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.589440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.616566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] MFA Rules not processed for user `61d9d17995cc4190b448625b13d82c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:54.617783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:54.627673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:54.640635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 940/1869] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:10:54 2022] POST /identity/v3/users => generated 293 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:10:54.644369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.656446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d597585-baa2-4b30-8148-9063465da92c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.698848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 930/1870] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:54.711774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.713469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:54.750362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.751134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.751519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.751864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.752678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.754506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.766942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 941/1871] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:10:54.774365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.786474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7b5a7-eb50-4cba-97ae-7d1a6845a496 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.792338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.794732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.795084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.795449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.796420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.798087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.798392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b19df-63b2-4036-8d4d-eaf281a8000b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.802225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9203704-7a63-4b1f-89e3-137d6391824b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.802917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9203704-7a63-4b1f-89e3-137d6391824b service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.803587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9203704-7a63-4b1f-89e3-137d6391824b service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.803929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9203704-7a63-4b1f-89e3-137d6391824b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.812575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 942/1872] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:10:54.813603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:54.816959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9203704-7a63-4b1f-89e3-137d6391824b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.819297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9203704-7a63-4b1f-89e3-137d6391824b service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:54.834135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3dbf0a-812f-4d9b-93a2-bf1fa09756ed admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:54.837021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 931/1873] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:10:54.839760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:54.840777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:54.841122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:54.841601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] PATH_INFO: `/v3/projects/31d2fa2c9aaa422c8c164f3da62ad924/users/a2c7bb0fe9b349bc97f18cf33f89a6f4/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:54.857423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d2fa2c9aaa422c8c164f3da62ad924, user_id=a2c7bb0fe9b349bc97f18cf33f89a6f4, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:54.860253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:55.007446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 943/1874] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:10:54 2022] PUT /identity/v3/projects/31d2fa2c9aaa422c8c164f3da62ad924/users/a2c7bb0fe9b349bc97f18cf33f89a6f4/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:10:55.018075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:55.018455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:55.018889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:55.048135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] MFA Rules not processed for user `a2c7bb0fe9b349bc97f18cf33f89a6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:10:55.049602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:55.122631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 932/1875] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:10:55.135629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:55.190830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c18f-6b51-46b4-bfc6-8e7bec988759 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:55.195057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:55.195821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:55.196108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:55.196389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:55.204925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:55.206350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:55.219408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 944/1876] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:55.580337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:55.888681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:10:55.890064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:55.926090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95449c-7720-4b65-9cbb-3889fec2110b None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:10:55.931058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:10:55.931841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:10:55.932359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:10:55.932697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:10:55.934629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:10:56.028852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:10:56.031785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:10:56.044314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 933/1877] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:10:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:10:58.311406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:10:58.946183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:00.418116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:01.317017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:01.492114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:02.437097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:02.445572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d209f-4eb3-47b7-b29c-c11d57a8439b service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:02.449241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62747fb-3ee5-48d4-90cd-e8fdd2daa852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:02.449876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62747fb-3ee5-48d4-90cd-e8fdd2daa852 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:02.450160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62747fb-3ee5-48d4-90cd-e8fdd2daa852 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:02.450436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62747fb-3ee5-48d4-90cd-e8fdd2daa852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:02.452222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d62747fb-3ee5-48d4-90cd-e8fdd2daa852 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:02.515758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62747fb-3ee5-48d4-90cd-e8fdd2daa852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:02.517293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62747fb-3ee5-48d4-90cd-e8fdd2daa852 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:02.527610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 945/1878] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:02.825233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:03.048531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:04.968539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:06.151457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:06.241053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.305097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b520a18-8af1-45c8-94a4-7f8c0b2a6608 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.309713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.310597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.310887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.311197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] PATH_INFO: `/v3/users/d41b3a4426ae43138186d334085e5564` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.312401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41b3a4426ae43138186d334085e5564)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.314362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.574269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41b3a4426ae43138186d334085e5564'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:06.575374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41b3a4426ae43138186d334085e5564'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:06.582251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41b3a4426ae43138186d334085e5564'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:06.582981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41b3a4426ae43138186d334085e5564'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:06.717510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:06.718243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41b3a4426ae43138186d334085e5564'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:06.736196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 934/1879] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:06 2022] DELETE /identity/v3/users/d41b3a4426ae43138186d334085e5564 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:06.741514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.741808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.742090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.760073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:06.761389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:06.819369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 946/1880] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:06.829314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:06.830763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:06.889394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f07523-8408-47a5-87cc-ecb73aef5983 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:06.893130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:06.893783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:06.894075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:06.894366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:06.909120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:06.910576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:06.923908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 935/1881] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:07.504785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.508371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:07.526580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d9f15-9ffc-4f5d-ba78-893a1ca51a67 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.534205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.534205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.534205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.534205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] PATH_INFO: `/v3/projects/986a20135f124bc9a34129de3d7f280e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.535530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=986a20135f124bc9a34129de3d7f280e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.538022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:07.713875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986a20135f124bc9a34129de3d7f280e', 'request_id': 'req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:07.719504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986a20135f124bc9a34129de3d7f280e', 'request_id': 'req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:07.725713 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986a20135f124bc9a34129de3d7f280e', 'request_id': 'req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:07.851625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:07.856510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] The token cache is being invalidate because project 986a20135f124bc9a34129de3d7f280e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:07.856806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:07.857430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:07.858981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 947/1882] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:07 2022] DELETE /identity/v3/projects/986a20135f124bc9a34129de3d7f280e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:07.863001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:07.874521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f047ac-ce6b-45d0-b99b-f80544daf96f service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:07.933637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:07.934532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:07.934820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:07.935103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] PATH_INFO: `/v3/users/61d9d17995cc4190b448625b13d82c7b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:07.936384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d9d17995cc4190b448625b13d82c7b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:07.938313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:08.312773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d9d17995cc4190b448625b13d82c7b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.313473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d9d17995cc4190b448625b13d82c7b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.320332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d9d17995cc4190b448625b13d82c7b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.320997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d9d17995cc4190b448625b13d82c7b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.414230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0c610a5c-9773-476d-b8e4-3adbf347d5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d9d17995cc4190b448625b13d82c7b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.433540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 936/1883] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:07 2022] DELETE /identity/v3/users/61d9d17995cc4190b448625b13d82c7b => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:08.448155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:08.449641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:08.468346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a9cda-6fe1-4c5c-a39a-aa224f2ed3c3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:08.532952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:08.534646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:08.534944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:08.535125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] PATH_INFO: `/v3/users/a2c7bb0fe9b349bc97f18cf33f89a6f4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:08.536631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c7bb0fe9b349bc97f18cf33f89a6f4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:08.539711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:08.539982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:08.739783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:08.755403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c7bb0fe9b349bc97f18cf33f89a6f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.763566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c7bb0fe9b349bc97f18cf33f89a6f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.764669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c7bb0fe9b349bc97f18cf33f89a6f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.765285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c7bb0fe9b349bc97f18cf33f89a6f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.781526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c7bb0fe9b349bc97f18cf33f89a6f4'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:08.865002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 948/1884] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:08 2022] DELETE /identity/v3/users/a2c7bb0fe9b349bc97f18cf33f89a6f4 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:08.869593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:08.870057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:08.870168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:08.887732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:08.888977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:08.917056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:08.920485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:08.965292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 937/1885] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:08.975386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:08.976774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:09.006661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bd2dd-4908-4690-bf86-72b4d7388daf admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:09.011805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:09.012671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.012953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.013239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] PATH_INFO: `/v3/projects/8c332ceb21f3474b89bbcbe58020ac3b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.014497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c332ceb21f3474b89bbcbe58020ac3b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:09.016461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:09.034950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2c117-e2f0-423e-b837-c0df33754518 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:09.038835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4353dce-c3c2-4217-819f-f0328121e35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:09.039519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353dce-c3c2-4217-819f-f0328121e35c service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.039815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353dce-c3c2-4217-819f-f0328121e35c service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.040104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4353dce-c3c2-4217-819f-f0328121e35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.051453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4353dce-c3c2-4217-819f-f0328121e35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:09.053056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4353dce-c3c2-4217-819f-f0328121e35c service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:09.066734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 938/1886] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:09.319801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c332ceb21f3474b89bbcbe58020ac3b', 'request_id': 'req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:09.325488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c332ceb21f3474b89bbcbe58020ac3b', 'request_id': 'req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:09.331851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c332ceb21f3474b89bbcbe58020ac3b', 'request_id': 'req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:09.428745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] The token cache is being invalidate because project 8c332ceb21f3474b89bbcbe58020ac3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:09.429163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:09.430246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1ec0aa38-9fee-462e-9246-73b7a8cb6c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:09.432191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 949/1887] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:08 2022] DELETE /identity/v3/projects/8c332ceb21f3474b89bbcbe58020ac3b => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:09.437776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b2e9-f2b5-427b-8635-c81dcd6e2881 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.438299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b2e9-f2b5-427b-8635-c81dcd6e2881 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.439477 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b2e9-f2b5-427b-8635-c81dcd6e2881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.486298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b288b2e9-f2b5-427b-8635-c81dcd6e2881 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:09.486298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b288b2e9-f2b5-427b-8635-c81dcd6e2881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:09.558710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 939/1888] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:09.566869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c165b49-4cab-400b-856d-89ad48dab4c6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.567436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c165b49-4cab-400b-856d-89ad48dab4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.567530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c165b49-4cab-400b-856d-89ad48dab4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.586345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6c165b49-4cab-400b-856d-89ad48dab4c6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:09.587594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c165b49-4cab-400b-856d-89ad48dab4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:09.596522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 950/1889] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:09.601466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288b2e9-f2b5-427b-8635-c81dcd6e2881 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:09.611478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288b2e9-f2b5-427b-8635-c81dcd6e2881 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:09.616442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:09.617130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.617415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.617694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.618421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:09.618728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:09.620427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:09.632683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 940/1890] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:09.637033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c165b49-4cab-400b-856d-89ad48dab4c6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:09.647111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c165b49-4cab-400b-856d-89ad48dab4c6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:09.651841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:09.652522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.652811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.653091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.653793 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:09.655786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:09.721047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:09.723046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:09.737830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6330b5a-4bec-4e5a-a311-f641bb0e51a8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:09.743153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:09.746026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.746333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.746628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] PATH_INFO: `/v3/projects/31d2fa2c9aaa422c8c164f3da62ad924` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.747987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d2fa2c9aaa422c8c164f3da62ad924)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:09.750169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:09.754143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 951/1891] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:09 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:09.758820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:09.831877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f3f30-ff3e-412f-8769-f91dd260d6d4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:09.837837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:09.838583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:09.838869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:09.839213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:09.839918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:09.842003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:09.911548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d2fa2c9aaa422c8c164f3da62ad924', 'request_id': 'req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:09.918662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d2fa2c9aaa422c8c164f3da62ad924', 'request_id': 'req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:09.928455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d2fa2c9aaa422c8c164f3da62ad924', 'request_id': 'req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.003198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 952/1892] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:09 2022] POST /identity/v3/users => generated 291 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:10.004300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.006363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.018831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d402-ce67-4e0c-b820-199f2f06e07a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.075283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.076277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.076561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.076838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] PATH_INFO: `/v3/users/7f2a7fb5016841c68af106e5c860a364` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.078030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2a7fb5016841c68af106e5c860a364)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.079995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.102909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] The token cache is being invalidate because project 31d2fa2c9aaa422c8c164f3da62ad924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:10.102909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.102909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.102909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 941/1893] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:09 2022] DELETE /identity/v3/projects/31d2fa2c9aaa422c8c164f3da62ad924 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:10.102909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.103337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.109283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ff43d-88fa-4e82-be4e-c88cbf5c28d5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.114515 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57941d92-947a-47dc-9c8a-1e9ef6670639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.115851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57941d92-947a-47dc-9c8a-1e9ef6670639 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.116583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57941d92-947a-47dc-9c8a-1e9ef6670639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.116583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57941d92-947a-47dc-9c8a-1e9ef6670639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.117307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57941d92-947a-47dc-9c8a-1e9ef6670639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.118917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57941d92-947a-47dc-9c8a-1e9ef6670639 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.129466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 942/1894] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:10.130942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526571dd-9a01-40a8-86dd-052bb40766fe None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.131258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526571dd-9a01-40a8-86dd-052bb40766fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.131546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526571dd-9a01-40a8-86dd-052bb40766fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.151637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-526571dd-9a01-40a8-86dd-052bb40766fe None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:10.153304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-526571dd-9a01-40a8-86dd-052bb40766fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.163994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 943/1895] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:10.165339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526571dd-9a01-40a8-86dd-052bb40766fe None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.184158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526571dd-9a01-40a8-86dd-052bb40766fe None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.190467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bab2ff-0557-4847-b033-505673eab3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.191250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab2ff-0557-4847-b033-505673eab3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.191609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab2ff-0557-4847-b033-505673eab3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.191944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab2ff-0557-4847-b033-505673eab3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.192973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bab2ff-0557-4847-b033-505673eab3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.195161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bab2ff-0557-4847-b033-505673eab3c0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.213963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 944/1896] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:10.213963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97360005-ff11-4be1-a92b-bb6a5dbd15fa None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.213963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97360005-ff11-4be1-a92b-bb6a5dbd15fa None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.213963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97360005-ff11-4be1-a92b-bb6a5dbd15fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.234145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-97360005-ff11-4be1-a92b-bb6a5dbd15fa None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:10.235468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-97360005-ff11-4be1-a92b-bb6a5dbd15fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.244877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 945/1897] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:10.246022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97360005-ff11-4be1-a92b-bb6a5dbd15fa None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.255764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97360005-ff11-4be1-a92b-bb6a5dbd15fa None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.261077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.262037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.262467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.262790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] PATH_INFO: `/v3/projects/e099b1a462d9487fa88a62ddc18910c2/users/3832d1db816842529ab2792173c3a8a8/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.278909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e099b1a462d9487fa88a62ddc18910c2, user_id=3832d1db816842529ab2792173c3a8a8, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.282058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.285718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:10.330961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2a7fb5016841c68af106e5c860a364'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.339045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2a7fb5016841c68af106e5c860a364'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.339774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2a7fb5016841c68af106e5c860a364'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.340378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2a7fb5016841c68af106e5c860a364'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.354056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2a7fb5016841c68af106e5c860a364'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:10.374156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 946/1898] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:10 2022] PUT /identity/v3/projects/e099b1a462d9487fa88a62ddc18910c2/users/3832d1db816842529ab2792173c3a8a8/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:10.374980 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.376571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.441665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8d4d-988f-43fe-b38c-a2c21bb9d977 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.446698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e679a3a-d6cf-46a5-8429-7ffb36e3a6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.447400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679a3a-d6cf-46a5-8429-7ffb36e3a6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.447693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679a3a-d6cf-46a5-8429-7ffb36e3a6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.447982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679a3a-d6cf-46a5-8429-7ffb36e3a6c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.448717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679a3a-d6cf-46a5-8429-7ffb36e3a6c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:10.449032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679a3a-d6cf-46a5-8429-7ffb36e3a6c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.450635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679a3a-d6cf-46a5-8429-7ffb36e3a6c9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.463056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 947/1899] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:10.464331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff624373-5301-4450-b82b-1811e682911d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.464625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff624373-5301-4450-b82b-1811e682911d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.466043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff624373-5301-4450-b82b-1811e682911d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.491383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ff624373-5301-4450-b82b-1811e682911d None None] MFA Rules not processed for user `3832d1db816842529ab2792173c3a8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:10.492661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ff624373-5301-4450-b82b-1811e682911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.494862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 953/1900] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:10 2022] DELETE /identity/v3/users/7f2a7fb5016841c68af106e5c860a364 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:10.495391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.556997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64565d1-da40-4580-b8fb-214cfe794436 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.559718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 948/1901] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:10.561000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.561295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.561578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.562426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.563508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.563899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.564251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.565019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.567032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.578859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:10.580143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.588231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 949/1902] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:10.588900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.590130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.602162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6863-4754-4a84-b12d-b9fd4bcf2b36 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.608431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.609203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.609574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.610016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.610766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.612739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.682625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 954/1903] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/projects => generated 361 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:10.683664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.685325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:10.716833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 950/1904] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:10.716833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.749688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198aa24-9a8f-477c-9dd1-4d892c46bd5a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.792101 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d627e-12e9-444b-a66e-b54434497472 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.796334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.797098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.797527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.797818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.798680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.800628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.808907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.809577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.809867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.810153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.820206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.822022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.831821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 955/1905] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:10.832528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.842283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f81443-76f6-42c8-8df6-a091b51489af service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.847280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.847970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.848267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.848547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.849237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.852141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.914001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 951/1906] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/users => generated 281 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:10.917873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.929660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ec8a9-feb7-429f-8fa8-6aacb96f775a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.934614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.935310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.935599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.935876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.936608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.938276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.949620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 952/1907] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:10.954320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.963633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 956/1908] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:10 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:10.967422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:10.969027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027b26e-db1a-4e00-a0b1-1090d512fe69 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.976516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.977202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.977688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.977975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.978722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.979106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90813b89-7e4b-48ac-a450-440ef8d0d015 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:10.980409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:10.995102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:10.995838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:10.996131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:10.996417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:10.998048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:10.999710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.006912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 953/1909] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:11.008074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.011524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 957/1910] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:11.016141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.019233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ed03d-69fa-4fa7-9f64-008a027cd446 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.027394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a87648-3c22-4f12-83f2-ae8207c04d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.028607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87648-3c22-4f12-83f2-ae8207c04d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.029049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87648-3c22-4f12-83f2-ae8207c04d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.032438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670169b3-d9c6-4801-bb6d-57ee0853d4e2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.032676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87648-3c22-4f12-83f2-ae8207c04d91 admin admin] PATH_INFO: `/v3/projects/a4184ccc175c4f459a11f5e05684bd15/users/ff69a9c8d84e4a19a1d6a585dc48255c/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.038487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.039744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.040291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.040722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] PATH_INFO: `/v3/projects/03325e7d084c4c08836c319e03ba4cc2/users/7490d2873e614eaa841ed6df2ab034ae/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.062075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] RBAC: Authorizing `identity:create_grant(project_id=03325e7d084c4c08836c319e03ba4cc2, user_id=7490d2873e614eaa841ed6df2ab034ae, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.065750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.070084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a87648-3c22-4f12-83f2-ae8207c04d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4184ccc175c4f459a11f5e05684bd15, user_id=ff69a9c8d84e4a19a1d6a585dc48255c, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.073119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a87648-3c22-4f12-83f2-ae8207c04d91 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.147790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 958/1911] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:11 2022] PUT /identity/v3/projects/03325e7d084c4c08836c319e03ba4cc2/users/7490d2873e614eaa841ed6df2ab034ae/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:11.156245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.185759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 954/1912] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:11 2022] PUT /identity/v3/projects/a4184ccc175c4f459a11f5e05684bd15/users/ff69a9c8d84e4a19a1d6a585dc48255c/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:11.191157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:11.200289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff381476-5796-4a3a-b177-2acf80668510 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.200859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff381476-5796-4a3a-b177-2acf80668510 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.201240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff381476-5796-4a3a-b177-2acf80668510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.237109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ff381476-5796-4a3a-b177-2acf80668510 None None] MFA Rules not processed for user `ff69a9c8d84e4a19a1d6a585dc48255c`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:11.238737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ff381476-5796-4a3a-b177-2acf80668510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:11.249410 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798799-1123-4f67-ab6b-b5f8d859d45a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.311019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.311731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.312020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.312299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.313035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.314705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.323205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 955/1913] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:11.327219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 959/1914] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:11.330115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff381476-5796-4a3a-b177-2acf80668510 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.347301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff381476-5796-4a3a-b177-2acf80668510 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.352288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.353197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.353484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.353763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] PATH_INFO: `/v3/projects/03325e7d084c4c08836c319e03ba4cc2/users/7490d2873e614eaa841ed6df2ab034ae/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.356313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03325e7d084c4c08836c319e03ba4cc2, user_id=7490d2873e614eaa841ed6df2ab034ae, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.357081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.359147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.359443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:11.373330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 956/1915] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:11 2022] PUT /identity/v3/projects/03325e7d084c4c08836c319e03ba4cc2/users/7490d2873e614eaa841ed6df2ab034ae/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:11.377494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.379385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:11.439524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d087905-4f74-45ab-882d-65b314a3d190 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.468996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8971751-4a4a-4f04-8e24-ad16ed63c652 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.473806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.474675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.474964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.475272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] PATH_INFO: `/v3/projects/c9b195ac106e40ebb3067c3b6c5f4d8d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.476485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b195ac106e40ebb3067c3b6c5f4d8d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.478409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.492755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d0996-1d1a-4b5e-86c9-7fc55bf242c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.493408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d0996-1d1a-4b5e-86c9-7fc55bf242c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.493693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d0996-1d1a-4b5e-86c9-7fc55bf242c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.493973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d0996-1d1a-4b5e-86c9-7fc55bf242c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.551420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d0996-1d1a-4b5e-86c9-7fc55bf242c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.553144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d0996-1d1a-4b5e-86c9-7fc55bf242c1 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.562377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 960/1916] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:11.563706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18c2f1-5231-4025-b4a4-e8c7eedf9dd4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.564001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18c2f1-5231-4025-b4a4-e8c7eedf9dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.564290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18c2f1-5231-4025-b4a4-e8c7eedf9dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.591065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0e18c2f1-5231-4025-b4a4-e8c7eedf9dd4 None None] MFA Rules not processed for user `7490d2873e614eaa841ed6df2ab034ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:11.592317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0e18c2f1-5231-4025-b4a4-e8c7eedf9dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:11.613264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b195ac106e40ebb3067c3b6c5f4d8d', 'request_id': 'req-9c228d11-feb6-4b21-9493-3d15433ad60c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.619448 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b195ac106e40ebb3067c3b6c5f4d8d', 'request_id': 'req-9c228d11-feb6-4b21-9493-3d15433ad60c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.627033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b195ac106e40ebb3067c3b6c5f4d8d', 'request_id': 'req-9c228d11-feb6-4b21-9493-3d15433ad60c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.663156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 961/1917] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:11.663966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18c2f1-5231-4025-b4a4-e8c7eedf9dd4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.674511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18c2f1-5231-4025-b4a4-e8c7eedf9dd4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.718302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] The token cache is being invalidate because project c9b195ac106e40ebb3067c3b6c5f4d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:11.718302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.718302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.718302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 957/1918] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:11 2022] DELETE /identity/v3/projects/c9b195ac106e40ebb3067c3b6c5f4d8d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:11.718302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.718648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:11.743123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.744453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.744895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.745334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] PATH_INFO: `/v3/users/ff69a9c8d84e4a19a1d6a585dc48255c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.746986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff69a9c8d84e4a19a1d6a585dc48255c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.750120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.791985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c228d11-feb6-4b21-9493-3d15433ad60c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.796282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.797948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.798245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.798534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.801393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:11.867034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.868553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.878376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 958/1919] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:11.879217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:11.880478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:11.898966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033016d9-5336-4452-8cc5-966af3be720b service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:11.904028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:11.907410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:11.907410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:11.907410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] PATH_INFO: `/v3/users/d1a70ded44cb4218b5d73d3451e9e5e3` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:11.907410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a70ded44cb4218b5d73d3451e9e5e3)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:11.908567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:11.980317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff69a9c8d84e4a19a1d6a585dc48255c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.987273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff69a9c8d84e4a19a1d6a585dc48255c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.988010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff69a9c8d84e4a19a1d6a585dc48255c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:11.988685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff69a9c8d84e4a19a1d6a585dc48255c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:12.001340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff69a9c8d84e4a19a1d6a585dc48255c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:12.029526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:12.121556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 962/1920] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:11 2022] DELETE /identity/v3/users/ff69a9c8d84e4a19a1d6a585dc48255c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:12.122519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:12.123997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:12.181288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c6e8b-8d0d-4fbb-a0f1-83461a7d7df7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:12.185087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e1826-ca78-44ae-8d74-7bb84270c5b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:12.185755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e1826-ca78-44ae-8d74-7bb84270c5b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:12.186044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e1826-ca78-44ae-8d74-7bb84270c5b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:12.186326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e1826-ca78-44ae-8d74-7bb84270c5b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:12.188156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ce1e1826-ca78-44ae-8d74-7bb84270c5b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:12.248127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e1826-ca78-44ae-8d74-7bb84270c5b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:12.249580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e1826-ca78-44ae-8d74-7bb84270c5b0 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:12.259572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 963/1921] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:12.260703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:12.261072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:12.261271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:12.279094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:12.280431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:12.283084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a70ded44cb4218b5d73d3451e9e5e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:12.283901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a70ded44cb4218b5d73d3451e9e5e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:12.292413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a70ded44cb4218b5d73d3451e9e5e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:12.293119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a70ded44cb4218b5d73d3451e9e5e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:12.351198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 964/1922] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:12.360832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:12.362177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:12.418659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4306c5e-c6c0-4dc5-9f35-c10eddb8a4e4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:12.422360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:12.423001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:12.423320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:12.423597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:12.432917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:12.434276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:12.443902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 965/1923] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:12.446343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a70ded44cb4218b5d73d3451e9e5e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:12.465440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 959/1924] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:11 2022] DELETE /identity/v3/users/d1a70ded44cb4218b5d73d3451e9e5e3 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:13.127757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:13.127757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:13.139267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d634e-d5e9-4ce4-888e-73693279f362 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:13.202524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:13.203431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:13.203716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:13.204005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] PATH_INFO: `/v3/users/3832d1db816842529ab2792173c3a8a8` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:13.205195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3832d1db816842529ab2792173c3a8a8)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:13.207135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:13.294371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:13.294371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:13.313537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a59b85-b77b-4bd0-9316-8780c6170dca admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:13.319441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:13.320586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:13.320965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:13.321249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] PATH_INFO: `/v3/projects/a4184ccc175c4f459a11f5e05684bd15` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:13.322947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4184ccc175c4f459a11f5e05684bd15)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:13.325147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:13.610347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3832d1db816842529ab2792173c3a8a8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.611553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4184ccc175c4f459a11f5e05684bd15', 'request_id': 'req-60a9b487-dc65-4da7-b240-e1792931c376'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.617759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3832d1db816842529ab2792173c3a8a8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.618131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4184ccc175c4f459a11f5e05684bd15', 'request_id': 'req-60a9b487-dc65-4da7-b240-e1792931c376'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.618460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3832d1db816842529ab2792173c3a8a8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.619093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3832d1db816842529ab2792173c3a8a8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.624584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4184ccc175c4f459a11f5e05684bd15', 'request_id': 'req-60a9b487-dc65-4da7-b240-e1792931c376'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.630943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3832d1db816842529ab2792173c3a8a8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.697805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] The token cache is being invalidate because project a4184ccc175c4f459a11f5e05684bd15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:13.698102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.698740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.701781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 960/1925] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:13 2022] DELETE /identity/v3/projects/a4184ccc175c4f459a11f5e05684bd15 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:13.702368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:13.703673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:13.767802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 966/1926] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:13 2022] DELETE /identity/v3/users/3832d1db816842529ab2792173c3a8a8 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:13.769072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9b487-dc65-4da7-b240-e1792931c376 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:13.771776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:13.773354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:13.827199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:13.828212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:13.828503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:13.828780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] PATH_INFO: `/v3/projects/de587413ae4e42e3a2ed796ab9c2cf9e` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:13.829995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de587413ae4e42e3a2ed796ab9c2cf9e)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:13.831945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:13.839797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985e886-e32d-443f-a4ad-77e87ac53127 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:13.845273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:13.850645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:13.850645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:13.850645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] PATH_INFO: `/v3/users/ef1599a5f0994d38987e0afc2530acfa` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:13.850645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1599a5f0994d38987e0afc2530acfa)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:13.850645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:13.970087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de587413ae4e42e3a2ed796ab9c2cf9e', 'request_id': 'req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.976175 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de587413ae4e42e3a2ed796ab9c2cf9e', 'request_id': 'req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:13.982290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de587413ae4e42e3a2ed796ab9c2cf9e', 'request_id': 'req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.088511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] The token cache is being invalidate because project de587413ae4e42e3a2ed796ab9c2cf9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:14.088812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.089727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f400aa65-fb91-4c8a-ad19-44cdd3b0b2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.091692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 961/1927] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:13 2022] DELETE /identity/v3/projects/de587413ae4e42e3a2ed796ab9c2cf9e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:14.097240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b09686-e6eb-49c9-a28b-28542b34e45d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.097543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b09686-e6eb-49c9-a28b-28542b34e45d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.097860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b09686-e6eb-49c9-a28b-28542b34e45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.103743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1599a5f0994d38987e0afc2530acfa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.111296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1599a5f0994d38987e0afc2530acfa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.113493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1599a5f0994d38987e0afc2530acfa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.113493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1599a5f0994d38987e0afc2530acfa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.117421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d5b09686-e6eb-49c9-a28b-28542b34e45d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.118665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d5b09686-e6eb-49c9-a28b-28542b34e45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.127913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-45fc2e74-1871-42c6-a5cf-74f28784c53b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1599a5f0994d38987e0afc2530acfa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:14.179236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 962/1928] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.180608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf35019-13f8-43ef-a71c-734ed732ea50 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.180999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf35019-13f8-43ef-a71c-734ed732ea50 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.181379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf35019-13f8-43ef-a71c-734ed732ea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.199551 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9bf35019-13f8-43ef-a71c-734ed732ea50 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.200893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9bf35019-13f8-43ef-a71c-734ed732ea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.211468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 963/1929] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.211468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d394-7e4d-4335-bc8d-f170294ca989 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.211468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d394-7e4d-4335-bc8d-f170294ca989 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.211468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d394-7e4d-4335-bc8d-f170294ca989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.232034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5662d394-7e4d-4335-bc8d-f170294ca989 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.233265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5662d394-7e4d-4335-bc8d-f170294ca989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.242043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 964/1930] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.243321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.243613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.243897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.262675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.263924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.272053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 965/1931] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.272838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.274061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.294554 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 967/1932] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:13 2022] DELETE /identity/v3/users/ef1599a5f0994d38987e0afc2530acfa => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:14.296169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a51b3-299e-49c9-b58f-85edaaa947fd None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.296625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a51b3-299e-49c9-b58f-85edaaa947fd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.297043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a51b3-299e-49c9-b58f-85edaaa947fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.317559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0e0a51b3-299e-49c9-b58f-85edaaa947fd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.318789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0e0a51b3-299e-49c9-b58f-85edaaa947fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.342007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35652d-7b30-4034-a642-6c5894487643 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.384592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 968/1933] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.390031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a51b3-299e-49c9-b58f-85edaaa947fd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.396248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a51b3-299e-49c9-b58f-85edaaa947fd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.396677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.397413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.397768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.398118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.401118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d70c4-4076-435f-b5c1-f0b72f539398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.401778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d70c4-4076-435f-b5c1-f0b72f539398 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.402064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d70c4-4076-435f-b5c1-f0b72f539398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.402343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d70c4-4076-435f-b5c1-f0b72f539398 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.403076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d70c4-4076-435f-b5c1-f0b72f539398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:14.403418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d70c4-4076-435f-b5c1-f0b72f539398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.405058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d70c4-4076-435f-b5c1-f0b72f539398 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.408078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.409578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.421407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 969/1934] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:14.421407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6506d39-de04-4608-9c06-d68035820b29 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.421407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6506d39-de04-4608-9c06-d68035820b29 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.421407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6506d39-de04-4608-9c06-d68035820b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.423895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 966/1935] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:14.424630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.435385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7950c2-1270-432a-bb42-b9f9ff536d66 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.438571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c6506d39-de04-4608-9c06-d68035820b29 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:14.439890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c6506d39-de04-4608-9c06-d68035820b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.440325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.441147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.441767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.442055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.442799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:14.443125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.445383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.453574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 970/1936] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:14.454452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6506d39-de04-4608-9c06-d68035820b29 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.467089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 967/1937] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:14.467336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6506d39-de04-4608-9c06-d68035820b29 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.468940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.472842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.473635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.474023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.475051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.476054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.478870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.483221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1df46c-2370-4927-b944-41906416581f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.489583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.490648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.491159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.493847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.495658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.497782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.639040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 971/1938] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:14.640915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.643007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 968/1939] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:14.643937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.644490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:14.669920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:14.738101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca483b-4d54-4b36-b0f7-38a26ed8ba61 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.790425 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68b5d3-3dad-4a98-afb1-da9f714a6332 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.807396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.808538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.809463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.809766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.810644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.814020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.815201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.815201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.815443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.815844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.842275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.845854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.860924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 972/1940] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:14.861860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.874286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07af548-cd3f-4c4e-920a-910d9fa89518 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.883119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.887819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.887819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.887819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.887819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.888282 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.952195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 969/1941] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:14.956356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:14.966791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c0cac-e6d9-433e-8559-e2177220b4a7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:14.971645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:14.972315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:14.972604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:14.972993 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:14.973735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:14.975352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:14.985444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 970/1942] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:14.989502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.000193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeb34c-1edc-45b3-bac2-d01a0679064d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.004857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.005500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.006411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.006889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.007645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.009238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.020864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 971/1943] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:15.025099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.035171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa9f0e-0bd7-4311-91f7-bb0c397d1240 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.040909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.041825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.042108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.042384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] PATH_INFO: `/v3/projects/d63d0dda45964a9badc7d1897ead09e7/users/8e3d8bedb0804162b5f20956022d4083/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.064281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63d0dda45964a9badc7d1897ead09e7, user_id=8e3d8bedb0804162b5f20956022d4083, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.064662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 973/1944] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:14 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:15.065795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.067701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.068350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.082114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949b406-bc6d-4c69-a2fe-dbb18913dbab admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.096076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.097149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.097529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.097883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] PATH_INFO: `/v3/projects/e099b1a462d9487fa88a62ddc18910c2` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.101775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e099b1a462d9487fa88a62ddc18910c2)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.103964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.202377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 972/1945] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:15 2022] PUT /identity/v3/projects/d63d0dda45964a9badc7d1897ead09e7/users/8e3d8bedb0804162b5f20956022d4083/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:15.203236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.272162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43fd6e-b6ea-4edf-b227-a669872de5f5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.277412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99623fd-56f8-434f-a72d-e28e92983337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.278132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99623fd-56f8-434f-a72d-e28e92983337 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.278410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99623fd-56f8-434f-a72d-e28e92983337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.278692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99623fd-56f8-434f-a72d-e28e92983337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.279500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99623fd-56f8-434f-a72d-e28e92983337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.286575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99623fd-56f8-434f-a72d-e28e92983337 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.293884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 973/1946] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:15.295703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cedc32-8a45-4ac0-b1ab-d8e24b79c5c3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.295999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cedc32-8a45-4ac0-b1ab-d8e24b79c5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.296281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cedc32-8a45-4ac0-b1ab-d8e24b79c5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.316140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-e2cedc32-8a45-4ac0-b1ab-d8e24b79c5c3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:15.317573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e2cedc32-8a45-4ac0-b1ab-d8e24b79c5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.330424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 974/1947] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:15.330424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cedc32-8a45-4ac0-b1ab-d8e24b79c5c3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.336564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cedc32-8a45-4ac0-b1ab-d8e24b79c5c3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.345079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e099b1a462d9487fa88a62ddc18910c2', 'request_id': 'req-5e553343-646b-421f-a185-1cb0d5502f0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.353632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e099b1a462d9487fa88a62ddc18910c2', 'request_id': 'req-5e553343-646b-421f-a185-1cb0d5502f0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.370244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e099b1a462d9487fa88a62ddc18910c2', 'request_id': 'req-5e553343-646b-421f-a185-1cb0d5502f0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.407254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.408142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.408512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.408869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.409687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.411655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.423258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 975/1948] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:15.424324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.482109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] The token cache is being invalidate because project e099b1a462d9487fa88a62ddc18910c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:15.482406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.483061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.487705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 974/1949] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:15 2022] DELETE /identity/v3/projects/e099b1a462d9487fa88a62ddc18910c2 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:15.487705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.490676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.492676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c853a-d04a-45ea-bc0b-e78d4141d013 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.496376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.497033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.497317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.497588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.500723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.508574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e553343-646b-421f-a185-1cb0d5502f0c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.513618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:15.515898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.517522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.518520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.519724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.520122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.520468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] PATH_INFO: `/v3/projects/dd93377e08d74db69ffe80d32ac46f04/users/9ace1fea90364b58887ea8dc74524515/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.529171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 976/1950] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:15.530099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.531595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.540843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd93377e08d74db69ffe80d32ac46f04, user_id=9ace1fea90364b58887ea8dc74524515, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.543694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.545281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8575b3-cbf5-4ee2-b3f6-4dab40566cf7 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.550876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.551829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.552164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.553140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] PATH_INFO: `/v3/projects/104f001638f64024bb131b1f59828ea3` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.554565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=104f001638f64024bb131b1f59828ea3)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.556742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.666354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 975/1951] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:15 2022] PUT /identity/v3/projects/dd93377e08d74db69ffe80d32ac46f04/users/9ace1fea90364b58887ea8dc74524515/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:15.667225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.668755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.701024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104f001638f64024bb131b1f59828ea3', 'request_id': 'req-2df14a04-898d-4e7e-bf5a-7781830fdbf2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.707439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104f001638f64024bb131b1f59828ea3', 'request_id': 'req-2df14a04-898d-4e7e-bf5a-7781830fdbf2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.716728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104f001638f64024bb131b1f59828ea3', 'request_id': 'req-2df14a04-898d-4e7e-bf5a-7781830fdbf2'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.763012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0a8e0-05d5-4b3b-850d-78f58c01d0f9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.809246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] The token cache is being invalidate because project 104f001638f64024bb131b1f59828ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:15.810999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.810999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2df14a04-898d-4e7e-bf5a-7781830fdbf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:15.812348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 977/1952] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:15 2022] DELETE /identity/v3/projects/104f001638f64024bb131b1f59828ea3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:15.813609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.813900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.814185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.822772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.823820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.824184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.824536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] PATH_INFO: `/v3/users/7490d2873e614eaa841ed6df2ab034ae` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.825848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7490d2873e614eaa841ed6df2ab034ae)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.827916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:15.844096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] MFA Rules not processed for user `9ace1fea90364b58887ea8dc74524515`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:15.845463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.938354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 978/1953] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:15.938354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:15.939797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:15.961218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c60644-5ea3-4ee4-be9a-3e290b9568dd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:15.966016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:15.967058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:15.967735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:15.967735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] PATH_INFO: `/v3/users/4ac713fd630d4d67ac6634552133da1f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:15.968823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac713fd630d4d67ac6634552133da1f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:15.970778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:16.032904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7490d2873e614eaa841ed6df2ab034ae'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.041974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7490d2873e614eaa841ed6df2ab034ae'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.042584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7490d2873e614eaa841ed6df2ab034ae'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.043278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7490d2873e614eaa841ed6df2ab034ae'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.056465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7490d2873e614eaa841ed6df2ab034ae'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.190972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 976/1954] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:15 2022] DELETE /identity/v3/users/7490d2873e614eaa841ed6df2ab034ae => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:16.192029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:16.193782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.259624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bdf3e-ac7a-445f-a10f-4e3ef0b889c9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:16.263674 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:16.264701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:16.265174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:16.265609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:16.267915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.329572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:16.331059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:16.343568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 977/1955] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:16.364058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac713fd630d4d67ac6634552133da1f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.364058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac713fd630d4d67ac6634552133da1f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.366279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:16.367895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.368918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac713fd630d4d67ac6634552133da1f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.369692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac713fd630d4d67ac6634552133da1f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.441157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf5e38-15d3-4b66-aa59-7176077897d0 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:16.447690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:16.449223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:16.449518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:16.449806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] PATH_INFO: `/v3/users/9ace1fea90364b58887ea8dc74524515` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:16.451306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ace1fea90364b58887ea8dc74524515)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:16.454787 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:16.488493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac713fd630d4d67ac6634552133da1f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.512484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 979/1956] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:15 2022] DELETE /identity/v3/users/4ac713fd630d4d67ac6634552133da1f => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:16.691226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ace1fea90364b58887ea8dc74524515'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.699965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ace1fea90364b58887ea8dc74524515'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.700774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ace1fea90364b58887ea8dc74524515'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.701618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ace1fea90364b58887ea8dc74524515'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.710539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:16.712383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.716329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ebea371f-0be1-4cf0-b99f-934f6b15789d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ace1fea90364b58887ea8dc74524515'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:16.804696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04168d-9564-46ae-a52c-4306b98140cf admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:16.809943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:16.811052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:16.811533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:16.811821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] PATH_INFO: `/v3/projects/03325e7d084c4c08836c319e03ba4cc2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:16.813245 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03325e7d084c4c08836c319e03ba4cc2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:16.815224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:16.832044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 978/1957] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:16 2022] DELETE /identity/v3/users/9ace1fea90364b58887ea8dc74524515 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:16.836466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:16.836761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:16.837046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:16.876728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:16.877951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:16.972784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 979/1958] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:16.982880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:16.984312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.020640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03325e7d084c4c08836c319e03ba4cc2', 'request_id': 'req-457e9fc7-d043-4124-bb5e-3c7068cf3723'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.028232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03325e7d084c4c08836c319e03ba4cc2', 'request_id': 'req-457e9fc7-d043-4124-bb5e-3c7068cf3723'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.034658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03325e7d084c4c08836c319e03ba4cc2', 'request_id': 'req-457e9fc7-d043-4124-bb5e-3c7068cf3723'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.045258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc0bfd-0d18-431a-853f-974699362aed None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.101291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66caec-d777-4039-9f37-77a3898a5124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.101987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66caec-d777-4039-9f37-77a3898a5124 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.102590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66caec-d777-4039-9f37-77a3898a5124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.102926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66caec-d777-4039-9f37-77a3898a5124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.167732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] The token cache is being invalidate because project 03325e7d084c4c08836c319e03ba4cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:17.168070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.168700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.169052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66caec-d777-4039-9f37-77a3898a5124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.170559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 980/1959] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:16 2022] DELETE /identity/v3/projects/03325e7d084c4c08836c319e03ba4cc2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:17.170667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66caec-d777-4039-9f37-77a3898a5124 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.171086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.172486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.181478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 980/1960] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:17.182763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8fcb4c-d9ba-4a08-94b1-01606dd8e1de None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.183057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8fcb4c-d9ba-4a08-94b1-01606dd8e1de None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.183368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8fcb4c-d9ba-4a08-94b1-01606dd8e1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.188008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e9fc7-d043-4124-bb5e-3c7068cf3723 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.192887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.194430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.194717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.194989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] PATH_INFO: `/v3/projects/2eb1a5c2d1cf4eed8166fb6516f379df` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.196304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb1a5c2d1cf4eed8166fb6516f379df)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.198205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.205832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9f8fcb4c-d9ba-4a08-94b1-01606dd8e1de None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:17.207228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9f8fcb4c-d9ba-4a08-94b1-01606dd8e1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.216406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 981/1961] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:17.225008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecb58c-70dc-43ad-9398-7ea0d7f7ea98 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.225909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecb58c-70dc-43ad-9398-7ea0d7f7ea98 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.226358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecb58c-70dc-43ad-9398-7ea0d7f7ea98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.245891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ffecb58c-70dc-43ad-9398-7ea0d7f7ea98 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:17.247348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ffecb58c-70dc-43ad-9398-7ea0d7f7ea98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.255765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 982/1962] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:17.260093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecb58c-70dc-43ad-9398-7ea0d7f7ea98 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.269368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecb58c-70dc-43ad-9398-7ea0d7f7ea98 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.274144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.274883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.275218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.275504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.276237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:17.276543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.278338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.290947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 983/1963] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:11:17.295220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.305997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5e6c7-74bf-49fb-90da-cc09a106987b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.310882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.311571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.311861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.312143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.312845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.314841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.352692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb1a5c2d1cf4eed8166fb6516f379df', 'request_id': 'req-0e9823aa-cc28-4d72-917b-1a5b753b588f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.358247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb1a5c2d1cf4eed8166fb6516f379df', 'request_id': 'req-0e9823aa-cc28-4d72-917b-1a5b753b588f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.367122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb1a5c2d1cf4eed8166fb6516f379df', 'request_id': 'req-0e9823aa-cc28-4d72-917b-1a5b753b588f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.429584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 984/1964] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:17.433731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.456625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] The token cache is being invalidate because project 2eb1a5c2d1cf4eed8166fb6516f379df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:17.457225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.458370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e9823aa-cc28-4d72-917b-1a5b753b588f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.467395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 981/1965] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:17 2022] DELETE /identity/v3/projects/2eb1a5c2d1cf4eed8166fb6516f379df => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:17.469013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cfc8d-e5d5-446c-8b0b-53f548b5a6ea None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.469335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cfc8d-e5d5-446c-8b0b-53f548b5a6ea None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.470183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cfc8d-e5d5-446c-8b0b-53f548b5a6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.498143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-686cfc8d-e5d5-446c-8b0b-53f548b5a6ea None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:17.499717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-686cfc8d-e5d5-446c-8b0b-53f548b5a6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.515385 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd36ce6-4cd8-4879-b95f-abe837769baa admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.521191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.521997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.522290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.522566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.523724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.525922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.573761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 982/1966] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:17.575203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958f929-a8a3-441b-a870-a04b809aa170 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.575519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958f929-a8a3-441b-a870-a04b809aa170 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.575805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958f929-a8a3-441b-a870-a04b809aa170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.595642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f958f929-a8a3-441b-a870-a04b809aa170 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:17.596866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f958f929-a8a3-441b-a870-a04b809aa170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.609043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 983/1967] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:17.609043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.609043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.609043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.635681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] MFA Rules not processed for user `8e3d8bedb0804162b5f20956022d4083`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:17.636891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.664127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 985/1968] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:17.665195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.676563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499419c-b939-40a3-9b56-3f42cce973f3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.683240 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.683949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.684205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.684490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.685229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:17.685813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.687928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.705170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 986/1969] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:17.706112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.707996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.721012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ee411-66bc-4444-a9ae-063d8398c47f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.727298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.733714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.733714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.733714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.733714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.734324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 984/1970] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:17.734432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.734526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.734526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.744621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 987/1971] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:17.745700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.749429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f242c-1413-43e7-bc21-6dc36e525f62 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.754033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.754893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.755234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.755518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] PATH_INFO: `/v3/projects/dd93377e08d74db69ffe80d32ac46f04` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.756532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e49e33-e9e1-4e51-a33a-69de613784b5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.757273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd93377e08d74db69ffe80d32ac46f04)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.759196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.761553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.762869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.763158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.763465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:17.764180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:17.766084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:17.882059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 988/1972] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:17 2022] POST /identity/v3/projects => generated 407 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:17.882059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:17.882059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:17.932528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd93377e08d74db69ffe80d32ac46f04', 'request_id': 'req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.939470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd93377e08d74db69ffe80d32ac46f04', 'request_id': 'req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.942719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b49ee-ba1a-417b-abbc-090cea4b3848 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:17.948940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd93377e08d74db69ffe80d32ac46f04', 'request_id': 'req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:17.994512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:17.995203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:17.995561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:17.995848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.037473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] The token cache is being invalidate because project dd93377e08d74db69ffe80d32ac46f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:18.042103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.042103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:18.044000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 985/1973] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:17 2022] DELETE /identity/v3/projects/dd93377e08d74db69ffe80d32ac46f04 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:18.044828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.049144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.062988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.064470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.075090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 989/1974] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:18.075900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.142193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:18.147455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50725f4-c21a-4a2b-9e54-5bc27aecfe32 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.150425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce30dbf-f56f-4b00-b0a0-9b1a911f0d91 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.152485 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b00d0-8993-45e3-ad30-70414ff8cd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.153315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b00d0-8993-45e3-ad30-70414ff8cd70 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.153728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b00d0-8993-45e3-ad30-70414ff8cd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.154138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b00d0-8993-45e3-ad30-70414ff8cd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.155085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b00d0-8993-45e3-ad30-70414ff8cd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.156145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d34944-693c-4408-81b9-e5a70f2fba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.156825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d34944-693c-4408-81b9-e5a70f2fba9c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.157022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b00d0-8993-45e3-ad30-70414ff8cd70 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.157158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d34944-693c-4408-81b9-e5a70f2fba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.157512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d34944-693c-4408-81b9-e5a70f2fba9c admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.158445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d34944-693c-4408-81b9-e5a70f2fba9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.160514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d34944-693c-4408-81b9-e5a70f2fba9c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.170440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 986/1975] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:18.172254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1468d-bfde-4d23-ba2c-348c1f1bc644 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.172735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1468d-bfde-4d23-ba2c-348c1f1bc644 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.173080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1468d-bfde-4d23-ba2c-348c1f1bc644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.194076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-94a1468d-bfde-4d23-ba2c-348c1f1bc644 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.195412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-94a1468d-bfde-4d23-ba2c-348c1f1bc644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.205599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 987/1976] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:18.206462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1468d-bfde-4d23-ba2c-348c1f1bc644 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.216917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1468d-bfde-4d23-ba2c-348c1f1bc644 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.223934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.225340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.225625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.225906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] PATH_INFO: `/v3/projects/133e28f0ef3045ab94532088179bff48/users/3da060cbc4854630ab5561de6b47b384/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.242547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=133e28f0ef3045ab94532088179bff48, user_id=3da060cbc4854630ab5561de6b47b384, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.245325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.268200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 990/1977] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/users => generated 304 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:18.268200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.268200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.268200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.286895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.288153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.299298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 991/1978] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:18.300182 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.301478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.313325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648f40a-1bad-44b2-a66b-b6a823c7b2f7 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.318073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.319525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.319912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.320305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.321301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.322968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.333269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 992/1979] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:18.334423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.347455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec7c5-f006-4142-af60-217e9ec2778b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.353688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 988/1980] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:18 2022] PUT /identity/v3/projects/133e28f0ef3045ab94532088179bff48/users/3da060cbc4854630ab5561de6b47b384/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:18.354808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.418850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d195344-e9c4-4e8e-8692-e6d6ae429a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.420361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195344-e9c4-4e8e-8692-e6d6ae429a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.420659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195344-e9c4-4e8e-8692-e6d6ae429a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.420940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d195344-e9c4-4e8e-8692-e6d6ae429a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.421737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d195344-e9c4-4e8e-8692-e6d6ae429a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:18.422205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d195344-e9c4-4e8e-8692-e6d6ae429a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.423881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d195344-e9c4-4e8e-8692-e6d6ae429a88 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.431514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a319b-4d04-4362-8bf2-184efd72efdd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.437091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.437775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.438064 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.438347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.439087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.440722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.450325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 993/1981] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:18.450325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af5501-8569-4a03-814b-6c72e998bffa None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.450325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af5501-8569-4a03-814b-6c72e998bffa None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.450325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af5501-8569-4a03-814b-6c72e998bffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.452229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 989/1982] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:18.453190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.464917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2423c05-121d-410b-9e1e-a96676c1449a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.469691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.470347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.470635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.470919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.471643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.473330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-50af5501-8569-4a03-814b-6c72e998bffa None None] MFA Rules not processed for user `3da060cbc4854630ab5561de6b47b384`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.473923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.475307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-50af5501-8569-4a03-814b-6c72e998bffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.541998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 994/1983] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:18.542742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af5501-8569-4a03-814b-6c72e998bffa None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.554262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af5501-8569-4a03-814b-6c72e998bffa None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.558963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.559895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.560186 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.560471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] PATH_INFO: `/v3/projects/b7a2735f98024ce8adbfda8ea66e16c2/users/2f96b6c54e2145e1b2c7a75eef093449/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.575102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a2735f98024ce8adbfda8ea66e16c2, user_id=2f96b6c54e2145e1b2c7a75eef093449, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.577911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.584451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 990/1984] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:18.584451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.584451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.642618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1f241-ca4c-4840-b53c-d0db9d6f612c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.646352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ba02a-7d8f-4664-a162-f2523bef32de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.647083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ba02a-7d8f-4664-a162-f2523bef32de service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.647409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ba02a-7d8f-4664-a162-f2523bef32de service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.647699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ba02a-7d8f-4664-a162-f2523bef32de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.671526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 995/1985] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:18 2022] PUT /identity/v3/projects/b7a2735f98024ce8adbfda8ea66e16c2/users/2f96b6c54e2145e1b2c7a75eef093449/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:18.672354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.706573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ba02a-7d8f-4664-a162-f2523bef32de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.708652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ba02a-7d8f-4664-a162-f2523bef32de service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.718880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 991/1986] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:18.720304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a192788-736f-4776-bcb9-2e4ac8e1cbbf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.720550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a192788-736f-4776-bcb9-2e4ac8e1cbbf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.720759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a192788-736f-4776-bcb9-2e4ac8e1cbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.738035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d9f09-7483-4ca1-909c-e96da146ca3e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.745725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.746456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.746753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.752030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.752030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.752308 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6a192788-736f-4776-bcb9-2e4ac8e1cbbf None None] MFA Rules not processed for user `2f96b6c54e2145e1b2c7a75eef093449`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:18.752400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.752497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6a192788-736f-4776-bcb9-2e4ac8e1cbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:18.820987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 992/1987] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:18.822721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a192788-736f-4776-bcb9-2e4ac8e1cbbf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.832786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a192788-736f-4776-bcb9-2e4ac8e1cbbf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.838392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.839063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.839383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.839670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.840493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.849680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 996/1988] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:18.851363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.869664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.881171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8b576-fbe4-47c8-94a6-44d19a0ccf4a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.886046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.886718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.887799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.887799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.888139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.889776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.901899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 997/1989] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:18.905790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.917424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd3b92-c36d-4931-876b-505fc595aed7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.922390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.923058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.923387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.923670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.924394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.926019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:18.937127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 998/1990] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:18.948126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.957682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480950cc-a012-4edd-b5d4-e6fa836ff40d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:18.963279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbff65-75e2-41d8-9c11-cd3312b6638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:18.964167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbff65-75e2-41d8-9c11-cd3312b6638a admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:18.967353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbff65-75e2-41d8-9c11-cd3312b6638a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:18.967353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbff65-75e2-41d8-9c11-cd3312b6638a admin admin] PATH_INFO: `/v3/projects/8c23292cfc7a4979a36523fd7f3cf427/users/61a08c3fdbc54dbe84d67a856aabdb26/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:18.972691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 993/1991] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/projects => generated 407 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:18.977822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:18.984071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbff65-75e2-41d8-9c11-cd3312b6638a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c23292cfc7a4979a36523fd7f3cf427, user_id=61a08c3fdbc54dbe84d67a856aabdb26, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:18.986987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbff65-75e2-41d8-9c11-cd3312b6638a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.047096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70b240-3c7a-46c2-a9d9-902244f71a3f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.051805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.052625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.052972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.053286 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.053986 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.055974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.134242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:19.140180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 999/1992] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:18 2022] PUT /identity/v3/projects/8c23292cfc7a4979a36523fd7f3cf427/users/61a08c3fdbc54dbe84d67a856aabdb26/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:19.150597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.150921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.151233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.176920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] MFA Rules not processed for user `61a08c3fdbc54dbe84d67a856aabdb26`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:19.178765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.239552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1000/1993] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:19.251198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.252522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.275639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 994/1994] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:18 2022] POST /identity/v3/users => generated 310 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:19.282115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.292462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978014d-c13e-45f1-841b-012bc325e436 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.311906 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a644b-af2f-4b94-a041-78ff8386dd65 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.315543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.319051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.319051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.319051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.326033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.327627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.337597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1001/1995] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:19.349688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.350578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.350870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.351149 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.351921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.353521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.365719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 995/1996] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:19.381340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.397175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63015ede-7805-4f87-b7d4-36324c405f8c service glance] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.408437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.409425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.410002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.410383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] PATH_INFO: `/v3/projects/ec9e11b814274bd9aee5867aff1066ba/users/9ed4acd370cb4fcfb52260500642cf7e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.428083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9e11b814274bd9aee5867aff1066ba, user_id=9ed4acd370cb4fcfb52260500642cf7e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.431282 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.578219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1002/1997] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:19 2022] PUT /identity/v3/projects/ec9e11b814274bd9aee5867aff1066ba/users/9ed4acd370cb4fcfb52260500642cf7e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:19.582139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.607792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2f832-71b5-4820-9b3c-9431d1b3bc08 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.663796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54161f6-c543-4948-b08f-4d24555f591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.668959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54161f6-c543-4948-b08f-4d24555f591c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.668959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54161f6-c543-4948-b08f-4d24555f591c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.668959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54161f6-c543-4948-b08f-4d24555f591c admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.668959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54161f6-c543-4948-b08f-4d24555f591c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.668959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54161f6-c543-4948-b08f-4d24555f591c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.680817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 996/1998] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:19.682212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.694671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74d0a4-175a-4fb8-8fb6-42546e3980e7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.699931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.700835 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.701127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.701409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] PATH_INFO: `/v3/projects/ec9e11b814274bd9aee5867aff1066ba/users/9ed4acd370cb4fcfb52260500642cf7e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.704724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9e11b814274bd9aee5867aff1066ba, user_id=9ed4acd370cb4fcfb52260500642cf7e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.706456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.718404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1003/1999] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:19 2022] PUT /identity/v3/projects/ec9e11b814274bd9aee5867aff1066ba/users/9ed4acd370cb4fcfb52260500642cf7e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:19.730620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1c714-29e8-4158-992d-a276c3708c7b None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.730986 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1c714-29e8-4158-992d-a276c3708c7b None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.731353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1c714-29e8-4158-992d-a276c3708c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.761392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-19c1c714-29e8-4158-992d-a276c3708c7b None None] MFA Rules not processed for user `9ed4acd370cb4fcfb52260500642cf7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:19.762724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-19c1c714-29e8-4158-992d-a276c3708c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:19.841051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 997/2000] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:19.852074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:19.911360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814de8c5-050e-4347-b5be-5df357de7a22 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:19.915245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:19.915904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:19.916191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:19.916473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:19.926963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:19.929189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:19.939829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1004/2001] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:20.033067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:20.033381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:20.033665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:20.059526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] MFA Rules not processed for user `8e3d8bedb0804162b5f20956022d4083`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:20.060806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:20.118314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 998/2002] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:20.434774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:20.444518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d820393-0446-4d9e-b5a1-ba373d8cae59 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:20.448545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:20.450254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:20.450254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:20.450254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:20.519799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:20.521843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:20.533970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1005/2003] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:20 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:21.448745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:22.408410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:24.105424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:24.477398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:24.622843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:24.624368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:24.706030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2357d-09b2-44e8-b8ab-ea771b28c0d9 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:24.711146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:24.712054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:24.712337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:24.712767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] PATH_INFO: `/v3/users/3dd72ff391dc471cb70da216dcdee69a` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:24.713861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd72ff391dc471cb70da216dcdee69a)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:24.715922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:24.988264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd72ff391dc471cb70da216dcdee69a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.988928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd72ff391dc471cb70da216dcdee69a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.996620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd72ff391dc471cb70da216dcdee69a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:24.997311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd72ff391dc471cb70da216dcdee69a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:25.206552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b483facb-e41d-48d8-803e-e968f6b32d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd72ff391dc471cb70da216dcdee69a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:25.225897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 999/2004] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:24 2022] DELETE /identity/v3/users/3dd72ff391dc471cb70da216dcdee69a => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:25.455120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:25.470908 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d819da8-be82-46cc-9197-2413e9966988 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:25.543905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:25.544763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:25.546420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:25.546420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] PATH_INFO: `/v3/projects/50f0e8f52e794a398b6b5db5634cf60a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:25.546420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f0e8f52e794a398b6b5db5634cf60a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:25.548383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:25.841197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f0e8f52e794a398b6b5db5634cf60a', 'request_id': 'req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:25.847369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f0e8f52e794a398b6b5db5634cf60a', 'request_id': 'req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:25.854021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f0e8f52e794a398b6b5db5634cf60a', 'request_id': 'req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.037736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] The token cache is being invalidate because project 50f0e8f52e794a398b6b5db5634cf60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:26.038096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.038827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4ceca5b7-b873-4e9f-a3b0-5ab4bbeb5876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:26.044869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1006/2005] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:25 2022] DELETE /identity/v3/projects/50f0e8f52e794a398b6b5db5634cf60a => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:26.046233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63724077-f2e7-404f-aa97-aa0e18418b31 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.046664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63724077-f2e7-404f-aa97-aa0e18418b31 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.046999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63724077-f2e7-404f-aa97-aa0e18418b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.069939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-63724077-f2e7-404f-aa97-aa0e18418b31 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:26.071147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-63724077-f2e7-404f-aa97-aa0e18418b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:26.141044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1000/2006] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:26.149278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804534-d0e0-4a30-814f-cf090f87ee91 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.149719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804534-d0e0-4a30-814f-cf090f87ee91 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.150071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804534-d0e0-4a30-814f-cf090f87ee91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.169633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c2804534-d0e0-4a30-814f-cf090f87ee91 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:26.170978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c2804534-d0e0-4a30-814f-cf090f87ee91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:26.179693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1007/2007] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:26.183933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63724077-f2e7-404f-aa97-aa0e18418b31 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.194753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63724077-f2e7-404f-aa97-aa0e18418b31 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.199750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.200966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.200966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.200966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.201717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:26.202034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.203672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.217007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1001/2008] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:26.220820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2804534-d0e0-4a30-814f-cf090f87ee91 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.230024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2804534-d0e0-4a30-814f-cf090f87ee91 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.234768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.235460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.235762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.236027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.236722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.239054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.400662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1008/2009] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:26 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:26.404405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.416456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec813b96-356f-4ae7-8fe9-d59b294ba9ea admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.480428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.481589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.481948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.482509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.483306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.486808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.635788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1002/2010] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:26 2022] POST /identity/v3/users => generated 282 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:26.640094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.669893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69675ce-a69a-4dbd-bf04-137661b14fa0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.677176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.677925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.678219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.678494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.679271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.681067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.699647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1009/2011] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:26.703936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.714649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869550cc-57f1-480e-a35f-7362cca21360 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.721167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b4d13-bfbd-489f-8139-014a2bfbc08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.721965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4d13-bfbd-489f-8139-014a2bfbc08f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.722236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4d13-bfbd-489f-8139-014a2bfbc08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.722589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4d13-bfbd-489f-8139-014a2bfbc08f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.723501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b4d13-bfbd-489f-8139-014a2bfbc08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.726285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b4d13-bfbd-489f-8139-014a2bfbc08f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.730287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:26.736948 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1003/2012] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:26.741330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:26.751538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890045-c4f5-4861-8960-2535cad5ca83 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:26.758795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:26.759834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.760050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.760333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] PATH_INFO: `/v3/projects/c57d0e5a60934b80aea58b264f5cfe25/users/48aad4aa64ab4adbb46054cff2d1ff73/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.783317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c57d0e5a60934b80aea58b264f5cfe25, user_id=48aad4aa64ab4adbb46054cff2d1ff73, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:26.789154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:26.888686 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1010/2013] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:26 2022] PUT /identity/v3/projects/c57d0e5a60934b80aea58b264f5cfe25/users/48aad4aa64ab4adbb46054cff2d1ff73/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:26.900981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde28c9c-ef0e-4a9e-9e1d-64a49c31b332 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:26.901217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde28c9c-ef0e-4a9e-9e1d-64a49c31b332 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:26.901301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde28c9c-ef0e-4a9e-9e1d-64a49c31b332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:26.935437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-bde28c9c-ef0e-4a9e-9e1d-64a49c31b332 None None] MFA Rules not processed for user `48aad4aa64ab4adbb46054cff2d1ff73`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:26.936585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bde28c9c-ef0e-4a9e-9e1d-64a49c31b332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:27.003778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1004/2014] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:27.009305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.072631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2094e8-5af3-4612-be0f-d8d0ce1206cc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.078916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.078916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.078916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.078916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.079499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.081439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.165983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1011/2015] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:27 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:27.169936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde28c9c-ef0e-4a9e-9e1d-64a49c31b332 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.182620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde28c9c-ef0e-4a9e-9e1d-64a49c31b332 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.214309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:27.247034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.247752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.248051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.248337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.249039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.250977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.339483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1005/2016] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:27 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:27.343451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.354681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efb7b0-0e6c-49ab-b4f5-e20c724ce40d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.359444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.360151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.360467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.360751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.361503 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.363195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.374300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1012/2017] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:27.378792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.389151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07ff0-69fe-4fd2-a5ad-41fc17a744a4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.393800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.394731 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.395017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.395331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] PATH_INFO: `/v3/projects/a33cf0f1fe5e4e22bee82d7bc513ba1a/users/d87b60f9f73c45c3906ce1ebe107e81b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.410670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33cf0f1fe5e4e22bee82d7bc513ba1a, user_id=d87b60f9f73c45c3906ce1ebe107e81b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.414414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.551397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1006/2018] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:27 2022] PUT /identity/v3/projects/a33cf0f1fe5e4e22bee82d7bc513ba1a/users/d87b60f9f73c45c3906ce1ebe107e81b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:27.555625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.565620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fa8ff-17e8-469f-b623-aa9d073913bc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.624553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdef57-ccc4-4c6b-ab22-48767d76784d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.625222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdef57-ccc4-4c6b-ab22-48767d76784d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.625509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdef57-ccc4-4c6b-ab22-48767d76784d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.625789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdef57-ccc4-4c6b-ab22-48767d76784d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.627590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdef57-ccc4-4c6b-ab22-48767d76784d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.628345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdef57-ccc4-4c6b-ab22-48767d76784d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.638792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1013/2019] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:27.642966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.654273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207b82-39bd-48cd-b31c-5a25eb0acf89 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.659110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.660053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.660338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.660618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] PATH_INFO: `/v3/projects/a33cf0f1fe5e4e22bee82d7bc513ba1a/users/d87b60f9f73c45c3906ce1ebe107e81b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.663251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33cf0f1fe5e4e22bee82d7bc513ba1a, user_id=d87b60f9f73c45c3906ce1ebe107e81b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.666327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.678066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1007/2020] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:27 2022] PUT /identity/v3/projects/a33cf0f1fe5e4e22bee82d7bc513ba1a/users/d87b60f9f73c45c3906ce1ebe107e81b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:27.689006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83040-bb71-4c99-8e96-fe9101221f7b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.689485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83040-bb71-4c99-8e96-fe9101221f7b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.696629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83040-bb71-4c99-8e96-fe9101221f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.715721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-05f83040-bb71-4c99-8e96-fe9101221f7b None None] MFA Rules not processed for user `d87b60f9f73c45c3906ce1ebe107e81b`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:27.716925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-05f83040-bb71-4c99-8e96-fe9101221f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:27.789369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1014/2021] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:27.801120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:27.803069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:27.864095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e75d7-02e7-40a3-991d-00e8a7e671be admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:27.869171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:27.869865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:27.870157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:27.870440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:27.883237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:27.883237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:27.892738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1008/2022] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:28.118692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f83040-bb71-4c99-8e96-fe9101221f7b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:28.128551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f83040-bb71-4c99-8e96-fe9101221f7b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:28.133041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd8090-253b-4f7f-a825-12c257cc8ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:28.133729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd8090-253b-4f7f-a825-12c257cc8ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:28.134022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd8090-253b-4f7f-a825-12c257cc8ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:28.134307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd8090-253b-4f7f-a825-12c257cc8ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:28.192721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd8090-253b-4f7f-a825-12c257cc8ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:28.194904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd8090-253b-4f7f-a825-12c257cc8ab4 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:28.203883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1015/2023] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:28.545325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:28.922394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:29.876786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:29.985348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:30.882612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:31.087024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:31.334284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:31.796631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:31.899146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:32.408204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:32.562213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:33.031003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:33.431203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:33.979263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:34.229651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:34.301537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11a40-0ec5-4d4b-8ed0-8f73cfb8757c service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:34.308021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:34.308889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:34.309223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:34.309464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] PATH_INFO: `/v3/users/48aad4aa64ab4adbb46054cff2d1ff73` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:34.310721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48aad4aa64ab4adbb46054cff2d1ff73)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:34.312700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:34.345475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:34.402720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:34.512837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48aad4aa64ab4adbb46054cff2d1ff73'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.513823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48aad4aa64ab4adbb46054cff2d1ff73'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.521554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48aad4aa64ab4adbb46054cff2d1ff73'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.522295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48aad4aa64ab4adbb46054cff2d1ff73'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.630682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48aad4aa64ab4adbb46054cff2d1ff73'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:34.649325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1009/2024] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:34 2022] DELETE /identity/v3/users/48aad4aa64ab4adbb46054cff2d1ff73 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:34.654151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:34.654394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:34.654673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:34.673679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:34.674876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:34.736661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1016/2025] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:34.746623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:34.748157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:34.812508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c4add-4298-4fff-96b4-db3d39971d40 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:34.816528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:34.817183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:34.817489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:34.817828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:34.831860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:34.833381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:34.845680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1010/2026] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:35.454294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:35.455951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:35.492378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d0137-d65a-43ce-a0e1-3ff1d99b20d5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:35.510308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:35.511479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:35.512152 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:35.513657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] PATH_INFO: `/v3/projects/c57d0e5a60934b80aea58b264f5cfe25` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:35.514798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c57d0e5a60934b80aea58b264f5cfe25)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:35.517608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:35.568277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:35.628167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:35.648797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c57d0e5a60934b80aea58b264f5cfe25', 'request_id': 'req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:35.654991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c57d0e5a60934b80aea58b264f5cfe25', 'request_id': 'req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:35.665903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c57d0e5a60934b80aea58b264f5cfe25', 'request_id': 'req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:35.785005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] The token cache is being invalidate because project c57d0e5a60934b80aea58b264f5cfe25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:35.785395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:35.786513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:35.788822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1017/2027] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:35 2022] DELETE /identity/v3/projects/c57d0e5a60934b80aea58b264f5cfe25 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:35.793374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:35.803821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ba10f-2cf4-4b3a-8a40-090326a0b78d service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:35.869544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:35.870600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:35.870901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:35.871527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] PATH_INFO: `/v3/users/d87b60f9f73c45c3906ce1ebe107e81b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:35.873195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87b60f9f73c45c3906ce1ebe107e81b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:35.875225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:36.054086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:36.132946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87b60f9f73c45c3906ce1ebe107e81b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:36.133760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87b60f9f73c45c3906ce1ebe107e81b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:36.140847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87b60f9f73c45c3906ce1ebe107e81b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:36.144841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87b60f9f73c45c3906ce1ebe107e81b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:36.247733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-69630bb8-adc1-44fd-9987-c281a6329f8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87b60f9f73c45c3906ce1ebe107e81b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:36.268098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1011/2028] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:35 2022] DELETE /identity/v3/users/d87b60f9f73c45c3906ce1ebe107e81b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:36.676697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:37.042216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:37.115993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:37.118940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:37.142224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04d00b-68b8-4ef6-92a3-abbe5b7cf071 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:37.233327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:37.234247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:37.234535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:37.234970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] PATH_INFO: `/v3/projects/a33cf0f1fe5e4e22bee82d7bc513ba1a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:37.236124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33cf0f1fe5e4e22bee82d7bc513ba1a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:37.239299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:37.405136 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33cf0f1fe5e4e22bee82d7bc513ba1a', 'request_id': 'req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:37.410748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33cf0f1fe5e4e22bee82d7bc513ba1a', 'request_id': 'req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:37.418354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33cf0f1fe5e4e22bee82d7bc513ba1a', 'request_id': 'req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:37.536888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] The token cache is being invalidate because project a33cf0f1fe5e4e22bee82d7bc513ba1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:37.538120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:37.540177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7eb13a77-9ab9-4f48-b17a-d0c402805d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:37.542851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1018/2029] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:37 2022] DELETE /identity/v3/projects/a33cf0f1fe5e4e22bee82d7bc513ba1a => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:37.547915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375de4-deeb-4a41-ad63-db747ee3c8c8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:37.548447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375de4-deeb-4a41-ad63-db747ee3c8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:37.548739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375de4-deeb-4a41-ad63-db747ee3c8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:37.572143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-df375de4-deeb-4a41-ad63-db747ee3c8c8 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:37.573805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-df375de4-deeb-4a41-ad63-db747ee3c8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:37.648564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1012/2030] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:37.656018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8afa72-1397-4489-a0f1-7dedd0f95d28 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:37.656315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8afa72-1397-4489-a0f1-7dedd0f95d28 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:37.656602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8afa72-1397-4489-a0f1-7dedd0f95d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:37.675882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7a8afa72-1397-4489-a0f1-7dedd0f95d28 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:37.677049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7a8afa72-1397-4489-a0f1-7dedd0f95d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:37.685732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1019/2031] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:37.690034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df375de4-deeb-4a41-ad63-db747ee3c8c8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:37.705472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:37.706039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df375de4-deeb-4a41-ad63-db747ee3c8c8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:37.712004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:37.712725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:37.713291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:37.713577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:37.714321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:37.714632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:37.716601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:37.733853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1013/2032] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:37.737982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8afa72-1397-4489-a0f1-7dedd0f95d28 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:37.748378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8afa72-1397-4489-a0f1-7dedd0f95d28 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:37.753252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:37.753938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:37.754230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:37.754527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:37.755276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:37.758430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:37.878143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1020/2033] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:37 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:37.883247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:37.893499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df9060-275a-4cfb-a7fa-f4114c8d1f27 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:37.953342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:37.954028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:37.954321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:37.954603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:37.955349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:37.957322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.140041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1014/2034] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:37 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:38.150403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.160730 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863a832-3bc1-4d91-a69a-4b942e8de546 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:38.165914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:38.166580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.166867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.167304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.168202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:38.169855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.181750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1021/2035] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:38.186018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.197283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858a09-3b6b-41c3-bbce-3389b31e8c37 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:38.201949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179021ee-6cf7-47fa-9f00-fb7c9778a844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:38.202619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179021ee-6cf7-47fa-9f00-fb7c9778a844 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.202929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179021ee-6cf7-47fa-9f00-fb7c9778a844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.203238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179021ee-6cf7-47fa-9f00-fb7c9778a844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.203999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179021ee-6cf7-47fa-9f00-fb7c9778a844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:38.205674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179021ee-6cf7-47fa-9f00-fb7c9778a844 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.219714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1015/2036] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:38.220858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.230777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b834-3116-4425-8113-93d7be24459c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:38.235462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:38.236344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.236628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.236902 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] PATH_INFO: `/v3/projects/5c136e0b88e44bb8bed7cd748abeebd2/users/5968095aa84e4c289d4d4c97b13ad658/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.252298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c136e0b88e44bb8bed7cd748abeebd2, user_id=5968095aa84e4c289d4d4c97b13ad658, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:38.255362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.419559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1022/2037] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:38 2022] PUT /identity/v3/projects/5c136e0b88e44bb8bed7cd748abeebd2/users/5968095aa84e4c289d4d4c97b13ad658/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:38.431706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.432031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.432323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.459835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] MFA Rules not processed for user `5968095aa84e4c289d4d4c97b13ad658`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:38.461067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:38.476872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:38.525444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1016/2038] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:38.529952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.552191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.554396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:38.597108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c0fb8-f127-4744-b045-dfeb0f73c3b8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:38.602850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:38.603576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.603891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.604312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.605041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:38.607032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.626013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f676db-89d2-47b8-8078-6d415f1ec392 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:38.630704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:38.631635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.631926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.632209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] PATH_INFO: `/v3/users/2f96b6c54e2145e1b2c7a75eef093449` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.633434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f96b6c54e2145e1b2c7a75eef093449)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:38.635394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.741609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1023/2039] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:38 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:38.745480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.821880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95226c0-f0f2-4e27-bdb2-9c97f1a34c16 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:38.826675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:38.827384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.827673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.827950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.828643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:38.830563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.860951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f96b6c54e2145e1b2c7a75eef093449'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:38.861614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f96b6c54e2145e1b2c7a75eef093449'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:38.869147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f96b6c54e2145e1b2c7a75eef093449'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:38.869781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f96b6c54e2145e1b2c7a75eef093449'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:38.952140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1024/2040] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:38 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:38.955653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.957275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:38.969242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754702e-e18a-4223-a89c-dd6979b4442d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:38.973490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b5c0d240-e280-472d-bc00-9bd26744d617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f96b6c54e2145e1b2c7a75eef093449'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:38.978084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:38.978084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.978084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.978084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:38.978697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:38.978697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:38.992564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1025/2041] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:38.993378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1017/2042] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:38 2022] DELETE /identity/v3/users/2f96b6c54e2145e1b2c7a75eef093449 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:38.993527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:38.997583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:38.997881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:38.998171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.003491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c01c62-e889-4598-a73f-15fc77f266d5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:39.016318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:39.017553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:39.062405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:39.063335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:39.063626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:39.063914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] PATH_INFO: `/v3/projects/7e472615c3544310aecffa23d9e40df6/users/1c1ed437652240c3a444b02a1647220d/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.076890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1018/2043] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:39.081119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e472615c3544310aecffa23d9e40df6, user_id=1c1ed437652240c3a444b02a1647220d, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:39.082703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:39.086864 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:39.088232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:39.145252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580616b9-716b-4511-97bb-78e14cdb141e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:39.149544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:39.150192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:39.150479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:39.150759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.160098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:39.162466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:39.172144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1019/2044] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:39.190721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1026/2045] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:38 2022] PUT /identity/v3/projects/7e472615c3544310aecffa23d9e40df6/users/1c1ed437652240c3a444b02a1647220d/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:39.195227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:39.206214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:39.213113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e79883-d0f0-4d90-b50f-aa8b4c1c0aa2 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:39.275785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a79825-7545-4f37-8323-7148ff2f57ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:39.276470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79825-7545-4f37-8323-7148ff2f57ea admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:39.276796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79825-7545-4f37-8323-7148ff2f57ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:39.277084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a79825-7545-4f37-8323-7148ff2f57ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.277977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a79825-7545-4f37-8323-7148ff2f57ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:39.279654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a79825-7545-4f37-8323-7148ff2f57ea admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:39.291524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1020/2046] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:39.295598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:39.306001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e0bbb-9bb1-43a8-b3ba-34e582a79c24 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:39.310701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:39.311617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:39.311901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:39.312180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] PATH_INFO: `/v3/projects/7e472615c3544310aecffa23d9e40df6/users/1c1ed437652240c3a444b02a1647220d/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.314294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e472615c3544310aecffa23d9e40df6, user_id=1c1ed437652240c3a444b02a1647220d, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:39.317100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:39.328499 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1027/2047] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:39 2022] PUT /identity/v3/projects/7e472615c3544310aecffa23d9e40df6/users/1c1ed437652240c3a444b02a1647220d/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:39.340110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:39.340110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:39.340406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.369526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] MFA Rules not processed for user `1c1ed437652240c3a444b02a1647220d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:39.370833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:39.447841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1021/2048] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:39.455874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:39.458272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:39.472046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:39.532479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3046b-2fcb-43ca-b464-a477b1418c8f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:39.536444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:39.537261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:39.537552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:39.537840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.539746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:39.551772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:39.609178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:39.610636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:39.624775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1028/2049] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:39.793256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:39.794896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:39.863958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b55d5-7577-4200-a2a3-96a331faa113 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:39.869506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:39.870405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:39.870689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:39.870968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] PATH_INFO: `/v3/projects/b7a2735f98024ce8adbfda8ea66e16c2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:39.872246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a2735f98024ce8adbfda8ea66e16c2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:39.874145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:40.048518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a2735f98024ce8adbfda8ea66e16c2', 'request_id': 'req-ecab5b30-4003-4854-b810-53317cba0214'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.054224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a2735f98024ce8adbfda8ea66e16c2', 'request_id': 'req-ecab5b30-4003-4854-b810-53317cba0214'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.060733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a2735f98024ce8adbfda8ea66e16c2', 'request_id': 'req-ecab5b30-4003-4854-b810-53317cba0214'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.213462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] The token cache is being invalidate because project b7a2735f98024ce8adbfda8ea66e16c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:40.213759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.214416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.219287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1022/2050] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:39 2022] DELETE /identity/v3/projects/b7a2735f98024ce8adbfda8ea66e16c2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:40.221002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:40.234402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aeea2-773d-4086-a5c0-9af847dcd31e service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:40.298870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:40.299797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:40.300086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:40.300370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] PATH_INFO: `/v3/users/9ed4acd370cb4fcfb52260500642cf7e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:40.301847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed4acd370cb4fcfb52260500642cf7e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:40.305241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:40.455735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:40.580511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed4acd370cb4fcfb52260500642cf7e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.588142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed4acd370cb4fcfb52260500642cf7e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.588859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed4acd370cb4fcfb52260500642cf7e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.589472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed4acd370cb4fcfb52260500642cf7e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.601820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-127cb5d4-0407-4698-9a76-81bd2ea4deb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed4acd370cb4fcfb52260500642cf7e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:40.658582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:40.693914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1029/2051] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:40 2022] DELETE /identity/v3/users/9ed4acd370cb4fcfb52260500642cf7e => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:41.243259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:41.244748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:41.259563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab5b30-4003-4854-b810-53317cba0214 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:41.294254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:41.353206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:41.353206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.353206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.353206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] PATH_INFO: `/v3/projects/ec9e11b814274bd9aee5867aff1066ba` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.353572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9e11b814274bd9aee5867aff1066ba)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.353572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.448810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9e11b814274bd9aee5867aff1066ba', 'request_id': 'req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.454290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9e11b814274bd9aee5867aff1066ba', 'request_id': 'req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.461127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9e11b814274bd9aee5867aff1066ba', 'request_id': 'req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.465965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:41.562100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] The token cache is being invalidate because project ec9e11b814274bd9aee5867aff1066ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:41.562402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.563118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0e57aa98-2f94-4d98-bca2-d82a78b0cc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:41.564860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1023/2052] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:41 2022] DELETE /identity/v3/projects/ec9e11b814274bd9aee5867aff1066ba => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:41.570343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b8159-faea-406c-bc36-39bfc47d1e51 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.570724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b8159-faea-406c-bc36-39bfc47d1e51 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.571108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b8159-faea-406c-bc36-39bfc47d1e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.590710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-670b8159-faea-406c-bc36-39bfc47d1e51 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:41.591974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-670b8159-faea-406c-bc36-39bfc47d1e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:41.651397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1030/2053] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:41.659365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b57c8-0128-456d-90fb-4f7cddb4dc9d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.659677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b57c8-0128-456d-90fb-4f7cddb4dc9d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.659998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b57c8-0128-456d-90fb-4f7cddb4dc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.680977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-329b57c8-0128-456d-90fb-4f7cddb4dc9d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:41.682199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-329b57c8-0128-456d-90fb-4f7cddb4dc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:41.690194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1024/2054] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:41.695087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b8159-faea-406c-bc36-39bfc47d1e51 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:41.704912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b8159-faea-406c-bc36-39bfc47d1e51 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:41.710983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:41.711774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.712064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.712344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.713087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:41.713399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.715123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.727122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1031/2055] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:41.731486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b57c8-0128-456d-90fb-4f7cddb4dc9d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:41.741251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b57c8-0128-456d-90fb-4f7cddb4dc9d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:41.745816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:41.746462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.746748 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.747042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.747752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.749667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.843333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1025/2056] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:41 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:41.847264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:41.857148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883774cc-c85b-44ba-a22a-722a885fcaee admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:41.913104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:41.913871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:41.914158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:41.914445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:41.915142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:41.917089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:41.945876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:42.030807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1032/2057] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:41 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:42.035164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.046624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9ea91-752b-4d5b-bb5f-5f2089c49207 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.051778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.053119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.053574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.053855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.054620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.056274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.069176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1026/2058] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:42.073398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.084116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7870af2-c9ef-4a70-8bcc-0d7fe7ad33f1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.089046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356d6d-ea35-4a0f-8786-5ab3463a1757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.089706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356d6d-ea35-4a0f-8786-5ab3463a1757 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.089997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356d6d-ea35-4a0f-8786-5ab3463a1757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.100346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356d6d-ea35-4a0f-8786-5ab3463a1757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.100346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356d6d-ea35-4a0f-8786-5ab3463a1757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.100346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356d6d-ea35-4a0f-8786-5ab3463a1757 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.103476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1033/2059] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:42.107398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.117631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc766f5-84f9-4ca7-a122-ff99df621c86 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.122256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.123122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.123439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.123716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] PATH_INFO: `/v3/projects/59b1db983f7b4b9f940a279bdc1e89e4/users/e44a1b53bb38476c87be15cb405caff0/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.138340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b1db983f7b4b9f940a279bdc1e89e4, user_id=e44a1b53bb38476c87be15cb405caff0, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.141181 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.209795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1027/2060] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:42 2022] PUT /identity/v3/projects/59b1db983f7b4b9f940a279bdc1e89e4/users/e44a1b53bb38476c87be15cb405caff0/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:42.220578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e16d95-a364-4751-aa72-cb3d28361178 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.220871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e16d95-a364-4751-aa72-cb3d28361178 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.221224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e16d95-a364-4751-aa72-cb3d28361178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.249680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e4e16d95-a364-4751-aa72-cb3d28361178 None None] MFA Rules not processed for user `e44a1b53bb38476c87be15cb405caff0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:42.250907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e4e16d95-a364-4751-aa72-cb3d28361178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:42.322143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1034/2061] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:42.330486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.380104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:42.406956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db2de8-b05d-4141-88fc-22db4efc26af admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.413546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.414295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.414512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.414801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.415252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:42.415548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.417481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.512516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1028/2062] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:42.516791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e16d95-a364-4751-aa72-cb3d28361178 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.528905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e16d95-a364-4751-aa72-cb3d28361178 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.589377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.590139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.590429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.590712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.591483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.593406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.690789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1035/2063] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:42 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:42.692117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.702442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47eaf8-c9e6-41f1-bd44-afcf76a92d58 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.707191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.707868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.708159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.709054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.709855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.711507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.725709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1029/2064] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:42.728865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.739162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299282d8-1027-49ed-82c6-dc57830527e1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.743868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.744741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.745027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.745308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] PATH_INFO: `/v3/projects/5b62c26423324ccd95b359979ad3924e/users/a188ef39dd93429b8857080d14fc489a/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.760506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b62c26423324ccd95b359979ad3924e, user_id=a188ef39dd93429b8857080d14fc489a, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.763186 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.888636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1036/2065] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:42 2022] PUT /identity/v3/projects/5b62c26423324ccd95b359979ad3924e/users/a188ef39dd93429b8857080d14fc489a/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:42.892670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:42.903705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2ac76-e6a6-4fce-8fac-ec36bb152a83 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:42.942159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:42.984085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605e632-36b4-44a5-bbe7-a0d975528032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:42.984844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605e632-36b4-44a5-bbe7-a0d975528032 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:42.985139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605e632-36b4-44a5-bbe7-a0d975528032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:42.985407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605e632-36b4-44a5-bbe7-a0d975528032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:42.986160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605e632-36b4-44a5-bbe7-a0d975528032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:42.987801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605e632-36b4-44a5-bbe7-a0d975528032 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:42.998661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1030/2066] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:43.002824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:43.013781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd295283-7475-419e-8611-6bf8c01be45a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:43.018395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:43.019313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:43.019598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:43.019875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] PATH_INFO: `/v3/projects/5b62c26423324ccd95b359979ad3924e/users/a188ef39dd93429b8857080d14fc489a/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:43.022706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b62c26423324ccd95b359979ad3924e, user_id=a188ef39dd93429b8857080d14fc489a, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:43.025425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:43.036144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1037/2067] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:43 2022] PUT /identity/v3/projects/5b62c26423324ccd95b359979ad3924e/users/a188ef39dd93429b8857080d14fc489a/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:43.046376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e0794-df94-44a7-981f-3f63155d9bc3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:43.046674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e0794-df94-44a7-981f-3f63155d9bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:43.046958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e0794-df94-44a7-981f-3f63155d9bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:43.073278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ab3e0794-df94-44a7-981f-3f63155d9bc3 None None] MFA Rules not processed for user `a188ef39dd93429b8857080d14fc489a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:43.074497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ab3e0794-df94-44a7-981f-3f63155d9bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:43.150395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:43.157454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1031/2068] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:43.183030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:43.185113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:43.249568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58d90a-3bba-4d5a-a86c-e2b30f0a6374 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:43.253384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:43.254062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:43.254365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:43.254626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:43.264199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:43.266130 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:43.275945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1038/2069] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:43.453907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:43.566496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e0794-df94-44a7-981f-3f63155d9bc3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:43.582297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:43.585957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e0794-df94-44a7-981f-3f63155d9bc3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:43.591348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:43.592152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:43.592445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:43.592729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:43.661550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:43.663950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:43.676874 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1032/2070] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:44.693942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:44.796847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.270924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.407725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.493736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.521947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.607639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.722109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.802382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:45.922366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.145233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.208086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.362235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.395551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.556604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.705248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:46.865850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:47.829068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:47.830834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:47.842873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad4973-95d8-4ae4-9a41-d7cce798b365 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:47.846687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:47.847446 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:47.847776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:47.848099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:47.850303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:47.928091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:47.928091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:47.935601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1039/2071] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:53.766223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:53.767714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:53.843872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4b431-aabd-4504-87cc-8babe0757bfa service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:53.849722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:53.851020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:53.851350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:53.851636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] PATH_INFO: `/v3/users/61a08c3fdbc54dbe84d67a856aabdb26` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:53.853262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a08c3fdbc54dbe84d67a856aabdb26)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:53.855759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:53.982679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:53.993648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32040b33-f70d-478f-923d-4b9dfefa7ad9 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:53.999465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:54.000391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:54.000711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:54.005924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] PATH_INFO: `/v3/users/e44a1b53bb38476c87be15cb405caff0` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:54.005924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44a1b53bb38476c87be15cb405caff0)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:54.005924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:54.007616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:54.080438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a08c3fdbc54dbe84d67a856aabdb26'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.081105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a08c3fdbc54dbe84d67a856aabdb26'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.089077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a08c3fdbc54dbe84d67a856aabdb26'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.090083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a08c3fdbc54dbe84d67a856aabdb26'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.144679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e705772f-a39f-47af-9f60-9e37cb2940c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a08c3fdbc54dbe84d67a856aabdb26'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.164310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1033/2072] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:53 2022] DELETE /identity/v3/users/61a08c3fdbc54dbe84d67a856aabdb26 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:54.169957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:54.170251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:54.170534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:54.189111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:54.190378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:54.217071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:54.246658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44a1b53bb38476c87be15cb405caff0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.254607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1034/2073] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:54.256536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44a1b53bb38476c87be15cb405caff0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.257254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44a1b53bb38476c87be15cb405caff0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.258045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44a1b53bb38476c87be15cb405caff0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.264840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:54.266617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:54.280198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44a1b53bb38476c87be15cb405caff0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:54.330033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac65332-62ef-4bc1-92a0-76351d2a69a5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:54.334049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff3a34-d3bd-48e7-96ec-8adc3200769e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:54.334706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff3a34-d3bd-48e7-96ec-8adc3200769e service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:54.334996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff3a34-d3bd-48e7-96ec-8adc3200769e service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:54.335379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff3a34-d3bd-48e7-96ec-8adc3200769e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:54.337253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e7ff3a34-d3bd-48e7-96ec-8adc3200769e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:54.351106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff3a34-d3bd-48e7-96ec-8adc3200769e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:54.352620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff3a34-d3bd-48e7-96ec-8adc3200769e service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:54.362681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1035/2074] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:54.395776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1040/2075] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:53 2022] DELETE /identity/v3/users/e44a1b53bb38476c87be15cb405caff0 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:54.401348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:54.402340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:54.402340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:54.421581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:54.422809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:54.483863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1036/2076] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:54.494814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:54.496320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:54.565395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447e657-283d-4162-b7e3-e4126057c996 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:54.574389 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:54.575056 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:54.575387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:54.575697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:54.587474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:54.589550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:54.599929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1041/2077] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:11:55.012462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:55.014005 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:55.026323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ca45e-b3ac-420c-852e-513020d605f6 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:55.031310 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:55.032178 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.032470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.032752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] PATH_INFO: `/v3/projects/8c23292cfc7a4979a36523fd7f3cf427` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.033966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c23292cfc7a4979a36523fd7f3cf427)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:55.036204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:55.186176 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c23292cfc7a4979a36523fd7f3cf427', 'request_id': 'req-4316d855-91d3-4be3-85d3-7614efbe0fa5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.192092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c23292cfc7a4979a36523fd7f3cf427', 'request_id': 'req-4316d855-91d3-4be3-85d3-7614efbe0fa5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.198563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c23292cfc7a4979a36523fd7f3cf427', 'request_id': 'req-4316d855-91d3-4be3-85d3-7614efbe0fa5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.202456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:55.363418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] The token cache is being invalidate because project 8c23292cfc7a4979a36523fd7f3cf427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:55.366964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.366964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4316d855-91d3-4be3-85d3-7614efbe0fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.367811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1037/2078] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:55 2022] DELETE /identity/v3/projects/8c23292cfc7a4979a36523fd7f3cf427 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:55.371024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:55.372845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:55.386939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6ef44-ccc2-44d9-96f9-6e61a23d94e4 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:55.398272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad369f0-bee6-4aa7-b33b-cbcb22f945c9 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.398623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad369f0-bee6-4aa7-b33b-cbcb22f945c9 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.400961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad369f0-bee6-4aa7-b33b-cbcb22f945c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.421476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-bad369f0-bee6-4aa7-b33b-cbcb22f945c9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:55.422682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bad369f0-bee6-4aa7-b33b-cbcb22f945c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:55.452741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:55.453705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.454067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.454436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] PATH_INFO: `/v3/projects/59b1db983f7b4b9f940a279bdc1e89e4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.455732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b1db983f7b4b9f940a279bdc1e89e4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:55.457714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:55.500536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1038/2079] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:55.510317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4f30f-15b8-43de-90b3-b82d793eef40 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.510647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4f30f-15b8-43de-90b3-b82d793eef40 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.510909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4f30f-15b8-43de-90b3-b82d793eef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.530301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a9a4f30f-15b8-43de-90b3-b82d793eef40 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:55.531604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a9a4f30f-15b8-43de-90b3-b82d793eef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:55.541226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1039/2080] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:55.546462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4f30f-15b8-43de-90b3-b82d793eef40 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:55.560639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4f30f-15b8-43de-90b3-b82d793eef40 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:55.561815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:55.562637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.562992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.563410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.564217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:55.564605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:55.566268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:55.578532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1040/2081] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:55.582759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:55.592183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:55.593020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105bc6c-cdbe-4ccd-a01e-ebd29b7a7265 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:55.597865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:55.598567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.598915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.599440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.600205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:55.602201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:55.609431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b1db983f7b4b9f940a279bdc1e89e4', 'request_id': 'req-c73d03b8-a2b7-485b-aa17-c58151396f6e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.615449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b1db983f7b4b9f940a279bdc1e89e4', 'request_id': 'req-c73d03b8-a2b7-485b-aa17-c58151396f6e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.621800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b1db983f7b4b9f940a279bdc1e89e4', 'request_id': 'req-c73d03b8-a2b7-485b-aa17-c58151396f6e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.734560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] The token cache is being invalidate because project 59b1db983f7b4b9f940a279bdc1e89e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:55.734972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.735842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:55.737896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1042/2082] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:55 2022] DELETE /identity/v3/projects/59b1db983f7b4b9f940a279bdc1e89e4 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:55.743376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:55.744971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:55.748106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1041/2083] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:55 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:55.749434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:55.751261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:55.816795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d03b8-a2b7-485b-aa17-c58151396f6e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:55.821565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df235d9-1809-4836-b7a7-d7f3d84d491c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:55.822675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:55.823925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.824292 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.824573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] PATH_INFO: `/v3/users/a188ef39dd93429b8857080d14fc489a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.825793 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a188ef39dd93429b8857080d14fc489a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:55.826398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:55.827067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:55.827388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:55.827691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:55.827844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:55.828371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:55.830298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:56.041255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1042/2084] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:55 2022] POST /identity/v3/users => generated 287 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:56.045246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:56.058127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a188ef39dd93429b8857080d14fc489a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.058979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b9b65-8ad6-49e0-8be5-89c631b07bd2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:56.065317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:56.065633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a188ef39dd93429b8857080d14fc489a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.066117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.066326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a188ef39dd93429b8857080d14fc489a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.066455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.066681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.066991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a188ef39dd93429b8857080d14fc489a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.067580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:56.069845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:56.079623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5181d68c-6c52-420d-a4e9-b52dd1dfee99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a188ef39dd93429b8857080d14fc489a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.080467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1043/2085] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:56.084463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:56.086179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:56.098797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac70cd-a925-453b-93f3-9a690e33781c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:56.103882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:56.104513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.104799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.105078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.105798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:56.107407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:56.117972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1044/2086] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:56.122162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:56.133230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec4a25-8484-40a6-b7f6-16e06e154801 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:56.137915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:56.138797 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.139087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.139397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] PATH_INFO: `/v3/projects/e5f021fe083b45c28ad560c481f307cf/users/aa342d57ade24c91aa33a9c71447b846/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.154595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f021fe083b45c28ad560c481f307cf, user_id=aa342d57ade24c91aa33a9c71447b846, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:56.157359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:56.190573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1043/2087] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:55 2022] DELETE /identity/v3/users/a188ef39dd93429b8857080d14fc489a => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:56.282347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:56.295724 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1045/2088] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:56 2022] PUT /identity/v3/projects/e5f021fe083b45c28ad560c481f307cf/users/aa342d57ade24c91aa33a9c71447b846/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:56.299839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.300190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.300481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.319054 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:56.320493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:56.320915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:56.322858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:56.356435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:56.392259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1044/2089] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:56.398397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62609fd1-98ed-49c8-8172-0a215d3c22c2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:56.403443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:56.403653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:56.404336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.404621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.404897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] PATH_INFO: `/v3/users/3da060cbc4854630ab5561de6b47b384` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.405002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:56.406112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3da060cbc4854630ab5561de6b47b384)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:56.408073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:56.467643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb9477-e7e2-4c6e-94f8-8ff5b835876f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:56.472017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:56.472674 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.472979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.473203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.483614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:56.485086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:56.495477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1045/2090] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:56.753068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:56.754670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:56.769463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ce917-cb6e-49bb-855b-89888dd235d1 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:56.776271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:56.777153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:56.777445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:56.778534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] PATH_INFO: `/v3/projects/5b62c26423324ccd95b359979ad3924e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:56.779264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b62c26423324ccd95b359979ad3924e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:56.782081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:56.789920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3da060cbc4854630ab5561de6b47b384'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.790988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3da060cbc4854630ab5561de6b47b384'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.799331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3da060cbc4854630ab5561de6b47b384'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:56.800016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3da060cbc4854630ab5561de6b47b384'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:57.041581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7128e0f-7b4a-417b-858b-c513e981bbb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3da060cbc4854630ab5561de6b47b384'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:57.075518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b62c26423324ccd95b359979ad3924e', 'request_id': 'req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:57.095611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1046/2091] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:11:56 2022] DELETE /identity/v3/users/3da060cbc4854630ab5561de6b47b384 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:57.098925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b62c26423324ccd95b359979ad3924e', 'request_id': 'req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:57.100289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.101002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.101576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.110914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b62c26423324ccd95b359979ad3924e', 'request_id': 'req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:57.128089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:57.129309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:57.197753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1047/2092] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:57.207466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.209769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:57.236760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:57.260039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] The token cache is being invalidate because project 5b62c26423324ccd95b359979ad3924e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:57.260422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:57.261129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc7d22cd-cb30-4f4b-9dd6-46c45a054e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:57.262818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1046/2093] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:56 2022] DELETE /identity/v3/projects/5b62c26423324ccd95b359979ad3924e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:57.267529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba98ab2-b53f-4828-8134-9516eea3e1e6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.267873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba98ab2-b53f-4828-8134-9516eea3e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.268176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba98ab2-b53f-4828-8134-9516eea3e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.284335 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ce6f1-4c2f-47ed-8247-e5596d1447fe None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.287380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-aba98ab2-b53f-4828-8134-9516eea3e1e6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:57.288578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-aba98ab2-b53f-4828-8134-9516eea3e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:57.289517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.290483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.290954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.291470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.294510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:57.296578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1047/2094] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:57.304281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8449d3d-5fed-479b-8720-ecad1c1a2781 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.304575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8449d3d-5fed-479b-8720-ecad1c1a2781 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.304997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8449d3d-5fed-479b-8720-ecad1c1a2781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.311665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.313685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.323621 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b8449d3d-5fed-479b-8720-ecad1c1a2781 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:57.324825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b8449d3d-5fed-479b-8720-ecad1c1a2781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:57.329211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1048/2095] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:57.332680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1048/2096] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:57.337839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.350537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0356a-af06-4213-95da-533dfd07d5c2 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.355728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.356394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.362237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.362237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.362237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:57.362237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.362237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.372733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1049/2097] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:57.376996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8449d3d-5fed-479b-8720-ecad1c1a2781 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.387764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8449d3d-5fed-479b-8720-ecad1c1a2781 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.393075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.393892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.394207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.394585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.395206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.397407 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.506396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1049/2098] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:57 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:57.510269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.521479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2a4f-4645-4138-9d70-e146c4e143df admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.589627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.590651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.591090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.591586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.593086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.596100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.715985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1050/2099] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:57 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:57.720055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.731167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d9a34-6833-47df-b683-56b5206d6b58 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.736308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.737294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.737586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.738011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.739070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.740647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.754918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1050/2100] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:57.759653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.772479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839928eb-7930-4e9e-bf12-ffa78d22fe71 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.777360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.778072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.778360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.778641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.779423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.781026 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.794743 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1051/2101] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:57.796491 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.807426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde04aa3-b639-439d-afb8-07d54cdb5674 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.812022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b214b-3c4c-446d-88a1-def267b4eedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.812904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b214b-3c4c-446d-88a1-def267b4eedf admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.813188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b214b-3c4c-446d-88a1-def267b4eedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.813462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b214b-3c4c-446d-88a1-def267b4eedf admin admin] PATH_INFO: `/v3/projects/38cea058035c4eb2aa46e19136885ed5/users/530d74cdc0c645aa94dc416770f1bde4/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.831696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:57.831696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b214b-3c4c-446d-88a1-def267b4eedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cea058035c4eb2aa46e19136885ed5, user_id=530d74cdc0c645aa94dc416770f1bde4, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.832165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b214b-3c4c-446d-88a1-def267b4eedf admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.855156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:57.856969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:57.870398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15cbeb-05fc-4b2d-8bad-cd900964aa59 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:57.875218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:57.876099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:57.876387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:57.876673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] PATH_INFO: `/v3/projects/133e28f0ef3045ab94532088179bff48` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:57.877878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=133e28f0ef3045ab94532088179bff48)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:57.880154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:57.990668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1051/2102] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:57 2022] PUT /identity/v3/projects/38cea058035c4eb2aa46e19136885ed5/users/530d74cdc0c645aa94dc416770f1bde4/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:58.001472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.001768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.002082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.034379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] MFA Rules not processed for user `530d74cdc0c645aa94dc416770f1bde4`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:58.035687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.041418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133e28f0ef3045ab94532088179bff48', 'request_id': 'req-d807787e-e835-4f9b-9384-44818994fed5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:58.048452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133e28f0ef3045ab94532088179bff48', 'request_id': 'req-d807787e-e835-4f9b-9384-44818994fed5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:58.055579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133e28f0ef3045ab94532088179bff48', 'request_id': 'req-d807787e-e835-4f9b-9384-44818994fed5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:58.104692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1052/2103] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:11:58.116087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.117421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.176795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2281ce-d78b-46d0-b1c6-df4284bf8e49 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.181303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12500180-dc0b-4f98-9131-f45136ca2a56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.181957 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500180-dc0b-4f98-9131-f45136ca2a56 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.182247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500180-dc0b-4f98-9131-f45136ca2a56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.182523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500180-dc0b-4f98-9131-f45136ca2a56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.191096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12500180-dc0b-4f98-9131-f45136ca2a56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.192551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12500180-dc0b-4f98-9131-f45136ca2a56 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.201929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1053/2104] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:58.207100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] The token cache is being invalidate because project 133e28f0ef3045ab94532088179bff48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:11:58.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:58.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d807787e-e835-4f9b-9384-44818994fed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:11:58.210765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1052/2105] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:11:57 2022] DELETE /identity/v3/projects/133e28f0ef3045ab94532088179bff48 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:11:58.214620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af12e0-d05f-4db3-92be-5ae9b14dce68 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.214919 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af12e0-d05f-4db3-92be-5ae9b14dce68 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.215227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af12e0-d05f-4db3-92be-5ae9b14dce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.232926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-37af12e0-d05f-4db3-92be-5ae9b14dce68 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:58.234118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-37af12e0-d05f-4db3-92be-5ae9b14dce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.311141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1054/2106] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:58.319834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb93060-237a-498a-8359-85db97f099fe None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.320126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb93060-237a-498a-8359-85db97f099fe None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.320439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb93060-237a-498a-8359-85db97f099fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.340627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-afb93060-237a-498a-8359-85db97f099fe None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:58.341995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-afb93060-237a-498a-8359-85db97f099fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.354191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1053/2107] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:58.358088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af12e0-d05f-4db3-92be-5ae9b14dce68 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.368376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af12e0-d05f-4db3-92be-5ae9b14dce68 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.374597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4709124-6e15-42bc-a726-e66c1bd65c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.377431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709124-6e15-42bc-a726-e66c1bd65c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.377431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709124-6e15-42bc-a726-e66c1bd65c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.377431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4709124-6e15-42bc-a726-e66c1bd65c92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.377431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4709124-6e15-42bc-a726-e66c1bd65c92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:11:58.377431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4709124-6e15-42bc-a726-e66c1bd65c92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.378539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4709124-6e15-42bc-a726-e66c1bd65c92 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.396970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1055/2108] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:11:58.411889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb93060-237a-498a-8359-85db97f099fe None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.411889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:11:58.412276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.412276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.412276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.413413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb93060-237a-498a-8359-85db97f099fe None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.418442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.420730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.421259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.421872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.422494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.425272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.469102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] MFA Rules not processed for user `aa342d57ade24c91aa33a9c71447b846`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:58.469102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.539804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1056/2109] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:58.552219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.553681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:58.565662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4bc3-f10d-4428-94ed-520017c4f748 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.570245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.571221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.571334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1054/2110] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:58 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:58.571439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.571553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.575234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.582817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.585161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.595877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1057/2111] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:11:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:11:58.638345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668afbc5-85a2-432d-8718-27f459080444 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.643114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.643818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.644113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.644397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.645092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.647639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.738754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1055/2112] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:58 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:58.743293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.753783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39fe5d-9ffa-4858-b628-a89c6573c3eb service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.759259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.759913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.760275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.760561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.761291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.762894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.773304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1058/2113] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:58.777716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.790883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7d745-35f0-41ae-9208-072e7aeed410 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.795913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab936e-003e-4ac6-a216-6f059bec7c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.796611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab936e-003e-4ac6-a216-6f059bec7c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.796936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab936e-003e-4ac6-a216-6f059bec7c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.797249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab936e-003e-4ac6-a216-6f059bec7c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.798022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab936e-003e-4ac6-a216-6f059bec7c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.799722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab936e-003e-4ac6-a216-6f059bec7c65 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.811127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1056/2114] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:58.815261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:58.826486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a6b42-b6ac-4401-8772-cc91ba249063 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:58.831572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:58.832658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.833017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.833421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] PATH_INFO: `/v3/projects/4aa34fa783d449559e8d5bc1f9c12b8d/users/8fc4483a7c87420c8ac75eb4afb14b87/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:58.848714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa34fa783d449559e8d5bc1f9c12b8d, user_id=8fc4483a7c87420c8ac75eb4afb14b87, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:58.852356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:58.976867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1059/2115] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:58 2022] PUT /identity/v3/projects/4aa34fa783d449559e8d5bc1f9c12b8d/users/8fc4483a7c87420c8ac75eb4afb14b87/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:58.993183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da7de3-7155-4866-8e0f-4b9479d1d666 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:58.993579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da7de3-7155-4866-8e0f-4b9479d1d666 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:58.993992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da7de3-7155-4866-8e0f-4b9479d1d666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.026962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-42da7de3-7155-4866-8e0f-4b9479d1d666 None None] MFA Rules not processed for user `8fc4483a7c87420c8ac75eb4afb14b87`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:59.028771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-42da7de3-7155-4866-8e0f-4b9479d1d666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:59.095798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1057/2116] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:59.100866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:59.167042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236604c-2f09-4dda-85e3-c4033b7733d7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:59.170624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:59.171258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:59.171602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:59.171884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.172601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:59.174633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:59.350909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1060/2117] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:59 2022] POST /identity/v3/projects => generated 383 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:11:59.354950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da7de3-7155-4866-8e0f-4b9479d1d666 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:59.362847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da7de3-7155-4866-8e0f-4b9479d1d666 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:59.419465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:59.420171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:59.420512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:59.420799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.421546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:59.423715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:59.542615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1058/2118] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:11:59 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:11:59.552622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:59.564670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55165-e3ee-4db9-9c3c-2fa3e3a1be02 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:59.570135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:59.570925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:59.571384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:59.571724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.572535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:59.574227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:59.587474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1061/2119] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:11:59.592044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:59.606113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5012e-dfab-43e8-8066-d6923992ed6d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:59.612102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a6da1-7abf-4a71-a8de-fd09f7f6b7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:59.613094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a6da1-7abf-4a71-a8de-fd09f7f6b7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:59.613597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a6da1-7abf-4a71-a8de-fd09f7f6b7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:59.614065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a6da1-7abf-4a71-a8de-fd09f7f6b7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.617052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a6da1-7abf-4a71-a8de-fd09f7f6b7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:59.619481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a6da1-7abf-4a71-a8de-fd09f7f6b7d2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:59.634495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1059/2120] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:11:59.638834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:59.649882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87df048-0ae2-4c69-878b-7d399bffc5af admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:59.656026 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:59.656951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:59.657265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:59.660474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] PATH_INFO: `/v3/projects/39cb5d18cd034de9b51635b17fdcd475/users/e061f3ca45e548f596ce8d61305c57ad/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.675635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39cb5d18cd034de9b51635b17fdcd475, user_id=e061f3ca45e548f596ce8d61305c57ad, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:59.679050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:11:59.796435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1062/2121] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:11:59 2022] PUT /identity/v3/projects/39cb5d18cd034de9b51635b17fdcd475/users/e061f3ca45e548f596ce8d61305c57ad/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:11:59.809044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849c35d-c83e-427a-bbc4-e864822035db None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:59.809525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849c35d-c83e-427a-bbc4-e864822035db None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:59.809989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849c35d-c83e-427a-bbc4-e864822035db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.840924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-2849c35d-c83e-427a-bbc4-e864822035db None None] MFA Rules not processed for user `e061f3ca45e548f596ce8d61305c57ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:11:59.842156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2849c35d-c83e-427a-bbc4-e864822035db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:11:59.906366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1060/2122] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:11:59.910394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:11:59.970811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f552016-a0d9-4ce6-8100-85cf34c47235 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:11:59.975530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:11:59.976188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:11:59.976474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:11:59.976756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:11:59.977455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:11:59.979398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:00.047999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:00.129122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1063/2123] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:11:59 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:00.133816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849c35d-c83e-427a-bbc4-e864822035db None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:00.144679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849c35d-c83e-427a-bbc4-e864822035db None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:00.203005 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:00.203744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:00.205702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:00.205702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:00.205702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:00.206877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:00.339781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1061/2124] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:00 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:00.343730 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:00.354602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c1ad1-48b0-4dcb-ab7d-ab2b8ef47510 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:00.359079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:00.359763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:00.360048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:00.360356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:00.361065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:00.362689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:00.374566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1064/2125] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:00.378757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:00.389265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0eef-588b-4d1b-98e5-64a9cb7d634c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:00.393883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:00.394747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:00.395030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:00.395356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] PATH_INFO: `/v3/projects/d8f700e629a4447a9cd0c5ff6f4e3acf/users/f3ef050063374009963cab563b3a6d0c/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:00.412248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f700e629a4447a9cd0c5ff6f4e3acf, user_id=f3ef050063374009963cab563b3a6d0c, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:00.414922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:00.534483 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:00.542326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1062/2126] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:00 2022] PUT /identity/v3/projects/d8f700e629a4447a9cd0c5ff6f4e3acf/users/f3ef050063374009963cab563b3a6d0c/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:00.547897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:00.560776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef09b-a2b6-47ef-a3be-24839a3ad810 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:00.621250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5186b-e7e2-48ad-997d-597ed6dac087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:00.621913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5186b-e7e2-48ad-997d-597ed6dac087 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:00.622202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5186b-e7e2-48ad-997d-597ed6dac087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:00.622481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5186b-e7e2-48ad-997d-597ed6dac087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:00.623247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5186b-e7e2-48ad-997d-597ed6dac087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:00.624827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5186b-e7e2-48ad-997d-597ed6dac087 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:00.634480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1065/2127] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:00.638838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:00.650123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9662-9769-44fb-a734-45ec6e37e28f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:00.654966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:00.655876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:00.656161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:00.656437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] PATH_INFO: `/v3/projects/d8f700e629a4447a9cd0c5ff6f4e3acf/users/f3ef050063374009963cab563b3a6d0c/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:00.658460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f700e629a4447a9cd0c5ff6f4e3acf, user_id=f3ef050063374009963cab563b3a6d0c, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:00.661453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:00.672491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1063/2128] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:00 2022] PUT /identity/v3/projects/d8f700e629a4447a9cd0c5ff6f4e3acf/users/f3ef050063374009963cab563b3a6d0c/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:00.683390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a50bf-24e4-437d-b1fb-4ad3423ed2e2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:00.683793 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a50bf-24e4-437d-b1fb-4ad3423ed2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:00.684047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a50bf-24e4-437d-b1fb-4ad3423ed2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:00.711758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cf2a50bf-24e4-437d-b1fb-4ad3423ed2e2 None None] MFA Rules not processed for user `f3ef050063374009963cab563b3a6d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:00.713023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cf2a50bf-24e4-437d-b1fb-4ad3423ed2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:00.783940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1066/2129] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:00.799421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:00.801113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:00.860203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e198e-9616-46f5-b662-2a0c7c84270f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:00.864083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:00.864782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:00.865112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:00.865400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:00.926728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:00.928419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:00.937386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1064/2130] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:00.967778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:01.321905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:01.604144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c4d1-9ccf-4f53-abdf-d0b553065078 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:01.604651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c4d1-9ccf-4f53-abdf-d0b553065078 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:01.604743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c4d1-9ccf-4f53-abdf-d0b553065078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:01.624111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d9b8c4d1-9ccf-4f53-abdf-d0b553065078 None None] MFA Rules not processed for user `aa342d57ade24c91aa33a9c71447b846`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:01.625306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d9b8c4d1-9ccf-4f53-abdf-d0b553065078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:01.684704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1067/2131] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:01.996721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:03.089387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:03.100817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52ca59-dd58-4a55-aa12-8c6abe60a8ca service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:03.105796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:03.106696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:03.106983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:03.107292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:03.167165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:03.168770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:03.178174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1065/2132] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:06.068397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c4d1-9ccf-4f53-abdf-d0b553065078 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:06.075522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c4d1-9ccf-4f53-abdf-d0b553065078 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:06.079083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f63d42-37ec-4a5f-825c-92c2e812104d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:06.079744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f63d42-37ec-4a5f-825c-92c2e812104d service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:06.080027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f63d42-37ec-4a5f-825c-92c2e812104d service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:06.080310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f63d42-37ec-4a5f-825c-92c2e812104d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:06.089482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f63d42-37ec-4a5f-825c-92c2e812104d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:06.090914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f63d42-37ec-4a5f-825c-92c2e812104d service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:06.100401 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1068/2133] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:07.311932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.313356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.379276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795f7a7-d46f-4965-802d-a2a6e5fbd416 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.384618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.385486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.385776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.386062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] PATH_INFO: `/v3/users/530d74cdc0c645aa94dc416770f1bde4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.387279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=530d74cdc0c645aa94dc416770f1bde4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.389260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:07.391602 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:07.640937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530d74cdc0c645aa94dc416770f1bde4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.641780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530d74cdc0c645aa94dc416770f1bde4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.651447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530d74cdc0c645aa94dc416770f1bde4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.652348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530d74cdc0c645aa94dc416770f1bde4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.764965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530d74cdc0c645aa94dc416770f1bde4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:07.787854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1066/2134] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:07 2022] DELETE /identity/v3/users/530d74cdc0c645aa94dc416770f1bde4 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:07.792571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.793431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.793722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.812509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:07.813970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.884218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1069/2135] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:07.894153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:07.896093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:07.975022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4148b-796f-4d3d-b767-b91dcced2fa1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:07.979803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d1e26-0b90-422f-a940-a76cc094e26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:07.980493 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d1e26-0b90-422f-a940-a76cc094e26e service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:07.980784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d1e26-0b90-422f-a940-a76cc094e26e service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:07.981070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d1e26-0b90-422f-a940-a76cc094e26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:07.990993 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d1e26-0b90-422f-a940-a76cc094e26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:07.993013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d1e26-0b90-422f-a940-a76cc094e26e service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:08.005034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1067/2136] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:08.644391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:08.646705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:08.661553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b16f5-e380-4045-a97d-b4b0892386ae None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:08.669830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:08.676101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:08.676101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:08.676101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] PATH_INFO: `/v3/projects/38cea058035c4eb2aa46e19136885ed5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:08.676101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38cea058035c4eb2aa46e19136885ed5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:08.676101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:08.809412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38cea058035c4eb2aa46e19136885ed5', 'request_id': 'req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:08.815457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38cea058035c4eb2aa46e19136885ed5', 'request_id': 'req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:08.821921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38cea058035c4eb2aa46e19136885ed5', 'request_id': 'req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:08.923376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] The token cache is being invalidate because project 38cea058035c4eb2aa46e19136885ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:08.923677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:08.924325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8e1bc5e7-3d96-4784-9a0f-e311cce3509d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:08.925856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1070/2137] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:08 2022] DELETE /identity/v3/projects/38cea058035c4eb2aa46e19136885ed5 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:08.931116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d960-8902-459c-a32d-c3bcb4e2eccd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:08.931635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d960-8902-459c-a32d-c3bcb4e2eccd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:08.931741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16d960-8902-459c-a32d-c3bcb4e2eccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:08.951418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6f16d960-8902-459c-a32d-c3bcb4e2eccd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:08.951970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6f16d960-8902-459c-a32d-c3bcb4e2eccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:09.027215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1068/2138] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:09.034831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0acc28-ac9f-43be-bc36-98cb4ee1ed7a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.035128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0acc28-ac9f-43be-bc36-98cb4ee1ed7a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.035429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0acc28-ac9f-43be-bc36-98cb4ee1ed7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.054071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bb0acc28-ac9f-43be-bc36-98cb4ee1ed7a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:09.055353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bb0acc28-ac9f-43be-bc36-98cb4ee1ed7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:09.065436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1071/2139] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:09.069438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16d960-8902-459c-a32d-c3bcb4e2eccd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:09.080838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16d960-8902-459c-a32d-c3bcb4e2eccd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:09.086820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:09.087574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.087866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.088146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.089609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:12:09.090094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:09.092018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:09.115992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1069/2140] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:12:09.117331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0acc28-ac9f-43be-bc36-98cb4ee1ed7a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:09.124584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:09.129178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0acc28-ac9f-43be-bc36-98cb4ee1ed7a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:09.152635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:09.153724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.154294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.156291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.157130 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:09.163671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:09.292840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1072/2141] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:09 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:09.297049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:09.309268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d3e6c-338d-4922-a093-88be4fa978d1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:09.324172 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:09.378055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:09.378735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.379024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.379513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.380039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:09.381976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:09.543503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1070/2142] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:09 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:09.544895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:09.555989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ca08e-6233-4f9e-bae3-e4d636012b26 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:09.560846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:09.561569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.561860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.562141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.563571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:09.565224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:09.577312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1073/2143] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:09.582192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:09.639903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab777db-b6ae-4a58-8e7a-2ddc46f5c1da admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:09.646532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ce0ed-ad46-4e5a-bf59-df2bc3f07b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:09.647127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ce0ed-ad46-4e5a-bf59-df2bc3f07b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.647531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ce0ed-ad46-4e5a-bf59-df2bc3f07b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.658164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ce0ed-ad46-4e5a-bf59-df2bc3f07b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.658164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ce0ed-ad46-4e5a-bf59-df2bc3f07b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:09.658164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ce0ed-ad46-4e5a-bf59-df2bc3f07b5f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:09.663555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1071/2144] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:09.666453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:09.678826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e90f2-0c3b-4091-ab1e-fae1de8bb700 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:09.686800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:09.687973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.688709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.689068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] PATH_INFO: `/v3/projects/3cdd5d3f0bc2431891a2fc795aeecc08/users/27650e513f3c4b8f8d2a4af9308ecb36/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.707701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdd5d3f0bc2431891a2fc795aeecc08, user_id=27650e513f3c4b8f8d2a4af9308ecb36, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:09.728233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:09.862895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1074/2145] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:09 2022] PUT /identity/v3/projects/3cdd5d3f0bc2431891a2fc795aeecc08/users/27650e513f3c4b8f8d2a4af9308ecb36/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:09.875979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae588fbf-dbd4-4288-9783-4d8cdf69d619 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:09.876276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae588fbf-dbd4-4288-9783-4d8cdf69d619 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:09.876561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae588fbf-dbd4-4288-9783-4d8cdf69d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:09.901802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ae588fbf-dbd4-4288-9783-4d8cdf69d619 None None] MFA Rules not processed for user `27650e513f3c4b8f8d2a4af9308ecb36`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:09.903058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ae588fbf-dbd4-4288-9783-4d8cdf69d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:09.968825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1072/2146] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:09.980209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:09.981894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:10.058722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef650f3-fff7-40b3-9a4d-6cbdf89ea044 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:10.063272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:10.064632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:10.064632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:10.064632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:10.076942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:10.078504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:10.088577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1075/2147] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:15.305460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb33d0c-832e-4702-bb2c-23f9f50e4f41 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:15.305997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb33d0c-832e-4702-bb2c-23f9f50e4f41 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:15.306341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb33d0c-832e-4702-bb2c-23f9f50e4f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:15.326201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9fb33d0c-832e-4702-bb2c-23f9f50e4f41 None None] MFA Rules not processed for user `27650e513f3c4b8f8d2a4af9308ecb36`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:15.327906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9fb33d0c-832e-4702-bb2c-23f9f50e4f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:15.341871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1073/2148] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:15.342070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:15.342070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:15.435632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5f28e-db23-49d0-863d-a4a9c328f307 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:15.441942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:15.443032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:15.443397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:15.443681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] PATH_INFO: `/v3/users/8fc4483a7c87420c8ac75eb4afb14b87` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:15.445084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc4483a7c87420c8ac75eb4afb14b87)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:15.447011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:15.748916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc4483a7c87420c8ac75eb4afb14b87'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:15.754387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:15.758077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc4483a7c87420c8ac75eb4afb14b87'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:15.758766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc4483a7c87420c8ac75eb4afb14b87'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:15.759411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc4483a7c87420c8ac75eb4afb14b87'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:15.772089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc4483a7c87420c8ac75eb4afb14b87'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:15.870536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1076/2149] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:15 2022] DELETE /identity/v3/users/8fc4483a7c87420c8ac75eb4afb14b87 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:15.875103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:15.875427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:15.875709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:15.892675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:15.893852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:15.970185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1074/2150] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:15.980306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:15.982167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:16.067536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381daa79-5b7e-4dcb-8916-2d090ca4f197 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:16.071431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:16.080684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:16.080684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:16.080684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:16.083591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:16.085839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:16.096883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1077/2151] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:16.628741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:16.635261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:16.657151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a87d-3563-4964-bb2e-c5982368897c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:16.663063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:16.664605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:16.664978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:16.665157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] PATH_INFO: `/v3/projects/4aa34fa783d449559e8d5bc1f9c12b8d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:16.666406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa34fa783d449559e8d5bc1f9c12b8d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:16.668568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:16.677405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:16.849697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa34fa783d449559e8d5bc1f9c12b8d', 'request_id': 'req-6a4e54b1-0e19-4085-948f-bc293dc3b25c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:16.856054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa34fa783d449559e8d5bc1f9c12b8d', 'request_id': 'req-6a4e54b1-0e19-4085-948f-bc293dc3b25c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:16.862746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa34fa783d449559e8d5bc1f9c12b8d', 'request_id': 'req-6a4e54b1-0e19-4085-948f-bc293dc3b25c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:16.938946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] The token cache is being invalidate because project 4aa34fa783d449559e8d5bc1f9c12b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:16.942532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:16.942532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:16.942532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1075/2152] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:16 2022] DELETE /identity/v3/projects/4aa34fa783d449559e8d5bc1f9c12b8d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:16.946069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:16.956134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd24021-74fc-4887-bdb5-6bd1a63a2729 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:17.017020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:17.017929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:17.021599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:17.021599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] PATH_INFO: `/v3/users/e061f3ca45e548f596ce8d61305c57ad` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:17.021599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e061f3ca45e548f596ce8d61305c57ad)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:17.021870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:17.299889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e061f3ca45e548f596ce8d61305c57ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:17.310352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e061f3ca45e548f596ce8d61305c57ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:17.311269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e061f3ca45e548f596ce8d61305c57ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:17.316821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e061f3ca45e548f596ce8d61305c57ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:17.329235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e061f3ca45e548f596ce8d61305c57ad'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:17.331074 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:17.426392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1078/2153] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:16 2022] DELETE /identity/v3/users/e061f3ca45e548f596ce8d61305c57ad => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:17.969401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:17.971611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:17.995166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e54b1-0e19-4085-948f-bc293dc3b25c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:18.064017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:18.064826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:18.065120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:18.065404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] PATH_INFO: `/v3/projects/39cb5d18cd034de9b51635b17fdcd475` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:18.066820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39cb5d18cd034de9b51635b17fdcd475)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:18.070833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:18.232348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39cb5d18cd034de9b51635b17fdcd475', 'request_id': 'req-91afa849-6c12-4ffe-95d5-39e0ea881250'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.238003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39cb5d18cd034de9b51635b17fdcd475', 'request_id': 'req-91afa849-6c12-4ffe-95d5-39e0ea881250'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.253121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39cb5d18cd034de9b51635b17fdcd475', 'request_id': 'req-91afa849-6c12-4ffe-95d5-39e0ea881250'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.374558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] The token cache is being invalidate because project 39cb5d18cd034de9b51635b17fdcd475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:18.377517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.377517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.377517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1076/2154] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:17 2022] DELETE /identity/v3/projects/39cb5d18cd034de9b51635b17fdcd475 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:18.381196 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:18.382781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:18.396897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501b0a0-1241-4e82-80b4-6f5a1dda9035 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:18.464672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:18.465599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:18.465836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:18.466119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] PATH_INFO: `/v3/users/f3ef050063374009963cab563b3a6d0c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:18.467367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ef050063374009963cab563b3a6d0c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:18.469395 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:18.803674 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ef050063374009963cab563b3a6d0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.810378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ef050063374009963cab563b3a6d0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.811039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ef050063374009963cab563b3a6d0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.811912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ef050063374009963cab563b3a6d0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.827394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f61bd36c-000f-4643-a2b1-9df4b98ca634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ef050063374009963cab563b3a6d0c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:18.945246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1079/2155] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:18 2022] DELETE /identity/v3/users/f3ef050063374009963cab563b3a6d0c => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:19.537601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:19.537601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:19.552778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afa849-6c12-4ffe-95d5-39e0ea881250 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:19.582352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:19.622855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:19.624428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:19.624744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:19.625093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] PATH_INFO: `/v3/projects/d8f700e629a4447a9cd0c5ff6f4e3acf` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:19.626523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f700e629a4447a9cd0c5ff6f4e3acf)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:19.630151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:19.783503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f700e629a4447a9cd0c5ff6f4e3acf', 'request_id': 'req-b795b54c-1338-4865-ac73-7d592f027eb7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:19.789677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f700e629a4447a9cd0c5ff6f4e3acf', 'request_id': 'req-b795b54c-1338-4865-ac73-7d592f027eb7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:19.795848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f700e629a4447a9cd0c5ff6f4e3acf', 'request_id': 'req-b795b54c-1338-4865-ac73-7d592f027eb7'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:19.926697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] The token cache is being invalidate because project d8f700e629a4447a9cd0c5ff6f4e3acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:19.927022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:19.927778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b795b54c-1338-4865-ac73-7d592f027eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:19.934206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1077/2156] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:19 2022] DELETE /identity/v3/projects/d8f700e629a4447a9cd0c5ff6f4e3acf => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:19.935960 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0a7c-6634-4eed-9efe-db4668208138 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:19.936430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0a7c-6634-4eed-9efe-db4668208138 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:19.936430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0a7c-6634-4eed-9efe-db4668208138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:19.964307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-39af0a7c-6634-4eed-9efe-db4668208138 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:19.965570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-39af0a7c-6634-4eed-9efe-db4668208138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:20.029442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1080/2157] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:20.037540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eebb8-b805-4816-b948-45ad14d414cc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.037849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eebb8-b805-4816-b948-45ad14d414cc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.038163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eebb8-b805-4816-b948-45ad14d414cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.066267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-008eebb8-b805-4816-b948-45ad14d414cc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:20.067408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-008eebb8-b805-4816-b948-45ad14d414cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:20.089192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1078/2158] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:20.094425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0a7c-6634-4eed-9efe-db4668208138 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:20.109961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0a7c-6634-4eed-9efe-db4668208138 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:20.117546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:20.118248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.118536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.118814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.119616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:12:20.119953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:20.123521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:20.130832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:20.138156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1081/2159] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:12:20.142580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eebb8-b805-4816-b948-45ad14d414cc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:20.152701 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eebb8-b805-4816-b948-45ad14d414cc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:20.158664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:20.159369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.160685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.161236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.161973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:20.164401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:20.421774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1079/2160] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:20 2022] POST /identity/v3/projects => generated 383 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:20.426073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:20.442052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d7823-b832-45cf-b73e-d0e93ac87fb1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:20.518300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:20.519025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.519347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.519631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.520375 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:20.523235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:20.649903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1082/2161] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:20 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:20.655666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:20.667793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1469008-b06a-4f4c-acc0-205f953ea1a2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:20.674426 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:20.675319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.675606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.675884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.676627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:20.678220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:20.689544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1080/2162] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:20.693671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:20.704347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99c28-51a6-44e8-a5d3-886ab8a0037a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:20.708707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05475bdb-d7d7-4021-b2bd-7096d6de55c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:20.709399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475bdb-d7d7-4021-b2bd-7096d6de55c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.709692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475bdb-d7d7-4021-b2bd-7096d6de55c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.709984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475bdb-d7d7-4021-b2bd-7096d6de55c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.711308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05475bdb-d7d7-4021-b2bd-7096d6de55c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:20.712907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05475bdb-d7d7-4021-b2bd-7096d6de55c9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:20.723153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1083/2163] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:20.727879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:20.738434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a9e89-00a6-449a-a7c6-8fb7f039591e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:20.744080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:20.744972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.745263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.745548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] PATH_INFO: `/v3/projects/db343c618fe842f58f7633fe7ec004e1/users/db455134fe1a48c7a7bd085bf788c431/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.761514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db343c618fe842f58f7633fe7ec004e1, user_id=db455134fe1a48c7a7bd085bf788c431, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:20.764243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:20.853891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1081/2164] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:20 2022] PUT /identity/v3/projects/db343c618fe842f58f7633fe7ec004e1/users/db455134fe1a48c7a7bd085bf788c431/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:20.864937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:20.865236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:20.865521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:20.891405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] MFA Rules not processed for user `db455134fe1a48c7a7bd085bf788c431`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:20.892689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:20.959977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1084/2165] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:21.249298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:21.251003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:21.255776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:21.258834 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:21.356437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd47f5-b5b7-4446-8271-35fd2e4a083e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:21.376765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1ba22-ac65-49e6-a872-d6c72b4268ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:21.378467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1ba22-ac65-49e6-a872-d6c72b4268ee service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:21.380669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1ba22-ac65-49e6-a872-d6c72b4268ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:21.381138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1ba22-ac65-49e6-a872-d6c72b4268ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:21.390867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c7d12-a2a4-4a70-81e0-49d75344ffac None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:21.402084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:21.402279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1ba22-ac65-49e6-a872-d6c72b4268ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:21.405138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1ba22-ac65-49e6-a872-d6c72b4268ee service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:21.405411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:21.406336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:21.406623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:21.406899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] PATH_INFO: `/v3/users/5968095aa84e4c289d4d4c97b13ad658` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:21.412269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5968095aa84e4c289d4d4c97b13ad658)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:21.416810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:21.424873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1082/2166] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:21.433187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:21.642961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5968095aa84e4c289d4d4c97b13ad658'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:21.652083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5968095aa84e4c289d4d4c97b13ad658'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:21.653237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5968095aa84e4c289d4d4c97b13ad658'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:21.654122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5968095aa84e4c289d4d4c97b13ad658'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:21.669985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5968095aa84e4c289d4d4c97b13ad658'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:21.796593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1085/2167] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:21 2022] DELETE /identity/v3/users/5968095aa84e4c289d4d4c97b13ad658 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:21.801273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:21.801762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:21.801841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:21.831229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:21.837282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:21.922562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1083/2168] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:21.934021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:21.942190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:22.049992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5d3de-4a14-4dd9-8c23-7e256afa119f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:22.055828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:22.056717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:22.058071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:22.058361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:22.071920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:22.075454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:22.085743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1086/2169] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:22.763992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:22.765484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:22.777914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92eae7-6e76-40a2-8d0b-75c028720065 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:22.782905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:22.784044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:22.784338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:22.784783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] PATH_INFO: `/v3/projects/5c136e0b88e44bb8bed7cd748abeebd2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:22.786268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c136e0b88e44bb8bed7cd748abeebd2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:22.788848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:23.055305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c136e0b88e44bb8bed7cd748abeebd2', 'request_id': 'req-a206c438-08b0-4bf3-9076-149c5a4942d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.066043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c136e0b88e44bb8bed7cd748abeebd2', 'request_id': 'req-a206c438-08b0-4bf3-9076-149c5a4942d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.073535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c136e0b88e44bb8bed7cd748abeebd2', 'request_id': 'req-a206c438-08b0-4bf3-9076-149c5a4942d4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.243646 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] The token cache is being invalidate because project 5c136e0b88e44bb8bed7cd748abeebd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:23.243941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.244839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.263926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1084/2170] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:22 2022] DELETE /identity/v3/projects/5c136e0b88e44bb8bed7cd748abeebd2 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:23.267159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:23.338819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ee23e-3a1b-4131-b806-7b0a144489c6 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:23.404420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:23.410347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:23.418107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:23.418107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:23.418107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] PATH_INFO: `/v3/users/1c1ed437652240c3a444b02a1647220d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:23.418107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1ed437652240c3a444b02a1647220d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:23.419474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:23.791575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1ed437652240c3a444b02a1647220d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.802348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1ed437652240c3a444b02a1647220d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.803949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1ed437652240c3a444b02a1647220d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.804949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1ed437652240c3a444b02a1647220d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.820297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1ed437652240c3a444b02a1647220d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:23.929368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1087/2171] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:23 2022] DELETE /identity/v3/users/1c1ed437652240c3a444b02a1647220d => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:24.636049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:24.638869 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:24.717187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206c438-08b0-4bf3-9076-149c5a4942d4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:24.744611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:24.778937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:24.779843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:24.780130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:24.780415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] PATH_INFO: `/v3/projects/7e472615c3544310aecffa23d9e40df6` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:24.781597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e472615c3544310aecffa23d9e40df6)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:24.783531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:24.860249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:24.861669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:24.874740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c5a54-9197-420e-a638-c50727871f6d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:24.879239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:24.880102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:24.880385 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:24.880678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] PATH_INFO: `/v3/users/db455134fe1a48c7a7bd085bf788c431` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:24.881868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db455134fe1a48c7a7bd085bf788c431)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:24.884350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:24.900489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e472615c3544310aecffa23d9e40df6', 'request_id': 'req-f49f6265-e47a-42c0-8a39-58a56b9b65ec'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:24.906312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e472615c3544310aecffa23d9e40df6', 'request_id': 'req-f49f6265-e47a-42c0-8a39-58a56b9b65ec'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:24.913027 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e472615c3544310aecffa23d9e40df6', 'request_id': 'req-f49f6265-e47a-42c0-8a39-58a56b9b65ec'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:24.996070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] The token cache is being invalidate because project 7e472615c3544310aecffa23d9e40df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:24.996396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:24.997062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f49f6265-e47a-42c0-8a39-58a56b9b65ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:24.998678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1085/2172] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:24 2022] DELETE /identity/v3/projects/7e472615c3544310aecffa23d9e40df6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:25.003638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bdcee-527e-4858-b1bb-bd0cc7cb69b5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.003971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bdcee-527e-4858-b1bb-bd0cc7cb69b5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.004313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bdcee-527e-4858-b1bb-bd0cc7cb69b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.057009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ce7bdcee-527e-4858-b1bb-bd0cc7cb69b5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:25.058229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ce7bdcee-527e-4858-b1bb-bd0cc7cb69b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:25.134044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1086/2173] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:25.143029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613ac4-c012-496b-8a79-337b35226e9d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.143206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613ac4-c012-496b-8a79-337b35226e9d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.143502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613ac4-c012-496b-8a79-337b35226e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.152168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db455134fe1a48c7a7bd085bf788c431'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:25.159451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db455134fe1a48c7a7bd085bf788c431'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:25.160336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db455134fe1a48c7a7bd085bf788c431'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:25.161274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db455134fe1a48c7a7bd085bf788c431'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:25.162814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-51613ac4-c012-496b-8a79-337b35226e9d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:25.164199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-51613ac4-c012-496b-8a79-337b35226e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:25.177227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1087/2174] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:25.178082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613ac4-c012-496b-8a79-337b35226e9d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.183847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-2f0c5eec-b184-4b32-ae03-2e9819e341d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db455134fe1a48c7a7bd085bf788c431'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:25.188795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613ac4-c012-496b-8a79-337b35226e9d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:25.193760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:25.194496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.195509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.195793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.196563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:12:25.196875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:25.198500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:25.210253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1088/2175] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:12:25.220217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.230049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ba018-eafa-40e5-8660-00ee3b6cef2d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:25.234793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:25.235482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.235781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.236061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.236749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:25.238766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:25.339653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1088/2176] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:24 2022] DELETE /identity/v3/users/db455134fe1a48c7a7bd085bf788c431 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:25.348880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.349177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.349462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.368318 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:25.369453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:25.428691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1089/2177] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:25.437521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1089/2178] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:25.438934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.440422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:25.442177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.506764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be4c3-e016-4e92-9047-e708b7cb95c0 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:25.512360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:25.513040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.513359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.513644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.514349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:25.516371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:25.542920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21eb435-94da-4754-afb3-84b01c5db202 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:25.596166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:25.596597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.596888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.597170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.607797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:25.609269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:25.624966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1090/2179] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:12:25.625095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1090/2180] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:25.626414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.639216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9364b4-1a59-4ce2-8a0c-2f2ed9ca4bfc service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:25.648861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:25.649826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.650281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.650720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.652066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:25.654124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:25.670042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1091/2181] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:25.674781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.688222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837484c-50c2-49ea-9383-6d4805ec2855 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:25.694283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dc76d-8634-4969-9513-4a444843f770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:25.695756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc76d-8634-4969-9513-4a444843f770 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.696285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc76d-8634-4969-9513-4a444843f770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.696672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dc76d-8634-4969-9513-4a444843f770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.697740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dc76d-8634-4969-9513-4a444843f770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:25.699973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dc76d-8634-4969-9513-4a444843f770 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:25.717962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.724652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1091/2182] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:25.730781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a593007-04be-4308-91ea-d5ce17ac0596 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:25.737330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b5592-7fa2-4020-a46b-f0436dadbadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:25.738535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b5592-7fa2-4020-a46b-f0436dadbadd admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.738886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b5592-7fa2-4020-a46b-f0436dadbadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.743430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b5592-7fa2-4020-a46b-f0436dadbadd admin admin] PATH_INFO: `/v3/projects/1616224747df4055969eef2cc1feaede/users/c9c518d5572e435c999b646d793e2b2f/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.755481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9586c-6a35-46f3-8d8d-c6896a36fd26 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.756269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9586c-6a35-46f3-8d8d-c6896a36fd26 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.758516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9586c-6a35-46f3-8d8d-c6896a36fd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.767866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b5592-7fa2-4020-a46b-f0436dadbadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1616224747df4055969eef2cc1feaede, user_id=c9c518d5572e435c999b646d793e2b2f, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:25.771873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b5592-7fa2-4020-a46b-f0436dadbadd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:25.791809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-09f9586c-6a35-46f3-8d8d-c6896a36fd26 None None] MFA Rules not processed for user `0fbb20d3c4bf4ed09a4e8c01ca10a927`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:25.794259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-09f9586c-6a35-46f3-8d8d-c6896a36fd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:25.850180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1092/2183] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:25 2022] PUT /identity/v3/projects/1616224747df4055969eef2cc1feaede/users/c9c518d5572e435c999b646d793e2b2f/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:25.856881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1092/2184] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:25.862529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca4e2c-b04a-400b-8ae2-3faef3f1ba6f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:25.863622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca4e2c-b04a-400b-8ae2-3faef3f1ba6f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:25.863622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca4e2c-b04a-400b-8ae2-3faef3f1ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:25.891020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0dca4e2c-b04a-400b-8ae2-3faef3f1ba6f None None] MFA Rules not processed for user `c9c518d5572e435c999b646d793e2b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:25.892458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0dca4e2c-b04a-400b-8ae2-3faef3f1ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:25.931023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:25.981411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1093/2185] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:25.982499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9586c-6a35-46f3-8d8d-c6896a36fd26 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:25.995307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9586c-6a35-46f3-8d8d-c6896a36fd26 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:26.061020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:26.061761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.062050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.062334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.063037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:26.064984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:26.190181 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1093/2186] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:25 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:26.191593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca4e2c-b04a-400b-8ae2-3faef3f1ba6f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:26.205174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca4e2c-b04a-400b-8ae2-3faef3f1ba6f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:26.271504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:26.272193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.272481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.274057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.274057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:26.276120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:26.442920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:26.443263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1094/2187] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:26 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:26.445903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:26.447352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:26.457947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b788b-90e0-4e6d-b87c-aebbeffe7200 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:26.461406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d647de-d5bc-443a-8d6a-2b009d8bfa6e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:26.463640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:26.464492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.464847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.465847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.465965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:26.467685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:26.474687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:26.475953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.476355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.476745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] PATH_INFO: `/v3/projects/db343c618fe842f58f7633fe7ec004e1` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.487377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db343c618fe842f58f7633fe7ec004e1)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:26.487377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:26.487857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1095/2188] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:26.489645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:26.501442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5324ef-d536-48a2-8c94-f7362cf26dc3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:26.506432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:26.508193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.508608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.508963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] PATH_INFO: `/v3/projects/ede58239854140f78b9366fed791b262/users/a3c4cd2145f942f88cd22b1a0ec13754/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.528774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede58239854140f78b9366fed791b262, user_id=a3c4cd2145f942f88cd22b1a0ec13754, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:26.531447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:26.691798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1096/2189] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:26 2022] PUT /identity/v3/projects/ede58239854140f78b9366fed791b262/users/a3c4cd2145f942f88cd22b1a0ec13754/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:26.696862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:26.705786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db343c618fe842f58f7633fe7ec004e1', 'request_id': 'req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:26.713180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db343c618fe842f58f7633fe7ec004e1', 'request_id': 'req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:26.726804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db343c618fe842f58f7633fe7ec004e1', 'request_id': 'req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:26.802745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bc15a-d4a3-477b-a72f-fd210fa1192d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:26.871826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] The token cache is being invalidate because project db343c618fe842f58f7633fe7ec004e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:26.872190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:26.872928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:26.883226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1094/2190] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:26 2022] DELETE /identity/v3/projects/db343c618fe842f58f7633fe7ec004e1 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:26.888538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ffe52-cbda-4f83-a065-56cec4ef2b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:26.890092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ffe52-cbda-4f83-a065-56cec4ef2b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.890606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ffe52-cbda-4f83-a065-56cec4ef2b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.892176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ffe52-cbda-4f83-a065-56cec4ef2b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.893227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ffe52-cbda-4f83-a065-56cec4ef2b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:26.895244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ffe52-cbda-4f83-a065-56cec4ef2b96 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:26.914927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1097/2191] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:26.918202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:26.924998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:26.939609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcfcd5-bef1-488d-bced-d8c0c7a30f8e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:26.946840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:26.951482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.951482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.951482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] PATH_INFO: `/v3/projects/ede58239854140f78b9366fed791b262/users/a3c4cd2145f942f88cd22b1a0ec13754/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:26.951482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede58239854140f78b9366fed791b262, user_id=a3c4cd2145f942f88cd22b1a0ec13754, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:26.954538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:26.976151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1095/2192] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:26 2022] PUT /identity/v3/projects/ede58239854140f78b9366fed791b262/users/a3c4cd2145f942f88cd22b1a0ec13754/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:26.981028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed4dc-6884-49bf-a371-390d91d9f933 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:26.981390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed4dc-6884-49bf-a371-390d91d9f933 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:26.981916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed4dc-6884-49bf-a371-390d91d9f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:27.019567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:27.021478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3eeed4dc-6884-49bf-a371-390d91d9f933 None None] MFA Rules not processed for user `a3c4cd2145f942f88cd22b1a0ec13754`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:27.022952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3eeed4dc-6884-49bf-a371-390d91d9f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:27.103156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1098/2193] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:27.113826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:27.115709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:27.175411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d4c5-23d3-4faa-8117-7276ded22337 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:27.179780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:27.180505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:27.180800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:27.181084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:27.191046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:27.192996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:27.204600 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1096/2194] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:27.317229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeed4dc-6884-49bf-a371-390d91d9f933 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:27.327744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeed4dc-6884-49bf-a371-390d91d9f933 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:27.331334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f45de0-435b-4958-b0ec-f89da12fec0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:27.331979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f45de0-435b-4958-b0ec-f89da12fec0c service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:27.332288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f45de0-435b-4958-b0ec-f89da12fec0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:27.332566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f45de0-435b-4958-b0ec-f89da12fec0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:27.334353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-19f45de0-435b-4958-b0ec-f89da12fec0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:27.397927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f45de0-435b-4958-b0ec-f89da12fec0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:27.399404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f45de0-435b-4958-b0ec-f89da12fec0c service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:27.409663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1099/2195] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:28.911135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:30.940511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:30.984319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:31.005662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3333-3d67-46c9-8fde-6c204d2c5430 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:31.011789 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:31.012683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:31.012973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:31.013278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] PATH_INFO: `/v3/users/c9c518d5572e435c999b646d793e2b2f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:31.014554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c518d5572e435c999b646d793e2b2f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:31.016532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:31.251067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c518d5572e435c999b646d793e2b2f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:31.251791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c518d5572e435c999b646d793e2b2f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:31.259406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c518d5572e435c999b646d793e2b2f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:31.260061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c518d5572e435c999b646d793e2b2f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:31.344303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c518d5572e435c999b646d793e2b2f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:31.363902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1097/2196] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:30 2022] DELETE /identity/v3/users/c9c518d5572e435c999b646d793e2b2f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:31.369230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac9a6c-7813-4af0-bb3c-547efdbc22b9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:31.369706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac9a6c-7813-4af0-bb3c-547efdbc22b9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:31.369963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac9a6c-7813-4af0-bb3c-547efdbc22b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:31.389072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c0ac9a6c-7813-4af0-bb3c-547efdbc22b9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:31.390275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c0ac9a6c-7813-4af0-bb3c-547efdbc22b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:31.451449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1100/2197] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:31.461092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:31.462675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:31.521932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd59f6-e6ea-49d3-90d4-7347ebde20d3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:31.525569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:31.526217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:31.526502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:31.526815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:31.536281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:31.537761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:31.548670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1098/2198] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:31.978901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:32.058398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c51c-582f-4171-84d8-851851e2508f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:32.058726 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c51c-582f-4171-84d8-851851e2508f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:32.059078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c51c-582f-4171-84d8-851851e2508f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:32.090354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d580c51c-582f-4171-84d8-851851e2508f None None] MFA Rules not processed for user `aa342d57ade24c91aa33a9c71447b846`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:32.093500 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d580c51c-582f-4171-84d8-851851e2508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:32.105312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:32.106976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:32.133767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea3eba-c885-41c6-8929-05a9283ffca8 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:32.151328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:32.152214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:32.152503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:32.153472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] PATH_INFO: `/v3/projects/1616224747df4055969eef2cc1feaede` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:32.155518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1616224747df4055969eef2cc1feaede)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:32.159729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:32.193222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1101/2199] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:32.323636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1616224747df4055969eef2cc1feaede', 'request_id': 'req-97419944-9abc-4fee-bb91-e763b7bcbb7c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.329165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1616224747df4055969eef2cc1feaede', 'request_id': 'req-97419944-9abc-4fee-bb91-e763b7bcbb7c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.335665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1616224747df4055969eef2cc1feaede', 'request_id': 'req-97419944-9abc-4fee-bb91-e763b7bcbb7c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.487343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] The token cache is being invalidate because project 1616224747df4055969eef2cc1feaede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:32.487644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.488293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.490080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1099/2200] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:32 2022] DELETE /identity/v3/projects/1616224747df4055969eef2cc1feaede => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:32.494639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c51c-582f-4171-84d8-851851e2508f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:32.506033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c51c-582f-4171-84d8-851851e2508f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:32.564968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:32.565854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:32.566134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:32.566411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] PATH_INFO: `/v3/users/a3c4cd2145f942f88cd22b1a0ec13754` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:32.567640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c4cd2145f942f88cd22b1a0ec13754)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:32.570228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:32.878205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c4cd2145f942f88cd22b1a0ec13754'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.885966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c4cd2145f942f88cd22b1a0ec13754'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.886632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c4cd2145f942f88cd22b1a0ec13754'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.887265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c4cd2145f942f88cd22b1a0ec13754'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:32.899492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-5b6fd115-dd3d-4bab-abf3-9df2da52c639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c4cd2145f942f88cd22b1a0ec13754'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:33.037576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1102/2201] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:32 2022] DELETE /identity/v3/users/a3c4cd2145f942f88cd22b1a0ec13754 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:33.552019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:33.560289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:33.568353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97419944-9abc-4fee-bb91-e763b7bcbb7c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:33.627755 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:33.628634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:33.628922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:33.629207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] PATH_INFO: `/v3/projects/ede58239854140f78b9366fed791b262` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:33.630435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede58239854140f78b9366fed791b262)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:33.633054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:33.786796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:33.787800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede58239854140f78b9366fed791b262', 'request_id': 'req-ef82b529-5078-4e0e-acd1-11ff256e3d33'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:33.794196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede58239854140f78b9366fed791b262', 'request_id': 'req-ef82b529-5078-4e0e-acd1-11ff256e3d33'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:33.800535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede58239854140f78b9366fed791b262', 'request_id': 'req-ef82b529-5078-4e0e-acd1-11ff256e3d33'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:33.938262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] The token cache is being invalidate because project ede58239854140f78b9366fed791b262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:33.938554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:33.939216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef82b529-5078-4e0e-acd1-11ff256e3d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:33.940760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1100/2202] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:33 2022] DELETE /identity/v3/projects/ede58239854140f78b9366fed791b262 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:33.946078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064773e1-5458-4dd5-adcc-6edefc3289cb None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:33.948522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064773e1-5458-4dd5-adcc-6edefc3289cb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:33.948522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064773e1-5458-4dd5-adcc-6edefc3289cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:33.966866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-064773e1-5458-4dd5-adcc-6edefc3289cb None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:33.968086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-064773e1-5458-4dd5-adcc-6edefc3289cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:34.027419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1103/2203] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:34.036561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9d55d-e818-4dd3-9e89-046b427459e0 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.036915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9d55d-e818-4dd3-9e89-046b427459e0 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.037151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9d55d-e818-4dd3-9e89-046b427459e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.059465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-58c9d55d-e818-4dd3-9e89-046b427459e0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:34.060690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-58c9d55d-e818-4dd3-9e89-046b427459e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:34.068999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1101/2204] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:34.073469 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064773e1-5458-4dd5-adcc-6edefc3289cb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:34.092562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064773e1-5458-4dd5-adcc-6edefc3289cb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:34.097588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:34.098252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.098569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.098888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.099682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:12:34.099996 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:34.101851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:34.118577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1104/2205] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:12:34.119478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9d55d-e818-4dd3-9e89-046b427459e0 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:34.128695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9d55d-e818-4dd3-9e89-046b427459e0 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:34.133568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:34.134220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.134503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.134783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.135529 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:34.137411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:34.347307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1102/2206] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:34 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:12:34.351985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:34.363462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de47254-4e46-439f-86cd-4af69990fbf0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:34.424953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:34.425636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.426001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.426294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.426991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:34.429200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:34.544179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1105/2207] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:34 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:34.545517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:34.555798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e12a5d-aab5-4f7f-adb8-a486569102a5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:34.560806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:34.561599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.562075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.562366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.563131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:34.578375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:34.578375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1103/2208] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:12:34.581253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:34.594951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7b9a0-7a68-4dde-a4e1-cd816c9fc4bd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:34.596876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559bfc0-6ba8-4558-bcf8-c766ea9dfffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:34.597751 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559bfc0-6ba8-4558-bcf8-c766ea9dfffd admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.598035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559bfc0-6ba8-4558-bcf8-c766ea9dfffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.598315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559bfc0-6ba8-4558-bcf8-c766ea9dfffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.599045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559bfc0-6ba8-4558-bcf8-c766ea9dfffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:34.600976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559bfc0-6ba8-4558-bcf8-c766ea9dfffd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:34.612162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1106/2209] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:34.616445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:34.626649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fff8f-6141-4661-abb2-85c07af4687f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:34.631651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:34.634229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.635311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.635671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] PATH_INFO: `/v3/projects/750892d71cea40bfb560b79c791a107b/users/e14b130e3e304044b1c39ba3c85aabfe/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.636915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:34.655144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750892d71cea40bfb560b79c791a107b, user_id=e14b130e3e304044b1c39ba3c85aabfe, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:34.657851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:34.796812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1104/2210] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:34 2022] PUT /identity/v3/projects/750892d71cea40bfb560b79c791a107b/users/e14b130e3e304044b1c39ba3c85aabfe/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:34.807585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0b86e-bce3-48bc-9b3b-6cc0b0544269 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.807815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0b86e-bce3-48bc-9b3b-6cc0b0544269 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.808099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0b86e-bce3-48bc-9b3b-6cc0b0544269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.835608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-13b0b86e-bce3-48bc-9b3b-6cc0b0544269 None None] MFA Rules not processed for user `e14b130e3e304044b1c39ba3c85aabfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:34.837236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-13b0b86e-bce3-48bc-9b3b-6cc0b0544269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:34.902836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1107/2211] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:34.907429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:34.970072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f52a6-cabf-47fa-9b6e-765889300620 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:34.974834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:34.976349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:34.976708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:34.976994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:34.977708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:34.979666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:35.187278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1105/2212] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:34 2022] POST /identity/v3/projects => generated 385 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:35.200203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0b86e-bce3-48bc-9b3b-6cc0b0544269 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:35.212728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0b86e-bce3-48bc-9b3b-6cc0b0544269 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:35.277025 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:35.277821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:35.278251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:35.278737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:35.279611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:35.281962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:35.488159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1108/2213] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:35 2022] POST /identity/v3/users => generated 299 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:35.492409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:35.502756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834341b-fbd4-4758-964f-d69e7331dde6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:35.508146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:35.508990 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:35.509347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:35.509699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:35.511144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:35.512942 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:35.528238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1106/2214] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:12:35.532278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:35.547561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551237f-291e-43fb-a003-5a26e85c6e18 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:35.552394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:35.553285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:35.553572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:35.554629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] PATH_INFO: `/v3/projects/648782cd11c14c55a2b5138094e6d2e4/users/2e90b7d8bdab40fdb1dddf6db4778308/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:35.571628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=648782cd11c14c55a2b5138094e6d2e4, user_id=2e90b7d8bdab40fdb1dddf6db4778308, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:35.574366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:35.718061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1109/2215] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:35 2022] PUT /identity/v3/projects/648782cd11c14c55a2b5138094e6d2e4/users/2e90b7d8bdab40fdb1dddf6db4778308/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:35.722613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:35.734671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c0065-0699-4c0b-8ecb-46c3ba75ec52 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:35.774062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:35.800373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe6915-d5fa-43c5-a45b-af72b67071be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:35.801207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe6915-d5fa-43c5-a45b-af72b67071be admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:35.801381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe6915-d5fa-43c5-a45b-af72b67071be admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:35.801663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe6915-d5fa-43c5-a45b-af72b67071be admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:35.802395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe6915-d5fa-43c5-a45b-af72b67071be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:35.804360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe6915-d5fa-43c5-a45b-af72b67071be admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:35.814586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1107/2216] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:35.818843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:35.836696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e6ba7-42a6-4465-892c-9b11d1674ab7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:35.841343 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:35.842226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:35.842514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:35.842795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] PATH_INFO: `/v3/projects/648782cd11c14c55a2b5138094e6d2e4/users/2e90b7d8bdab40fdb1dddf6db4778308/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:35.844824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=648782cd11c14c55a2b5138094e6d2e4, user_id=2e90b7d8bdab40fdb1dddf6db4778308, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:35.847698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:35.858536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1110/2217] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:35 2022] PUT /identity/v3/projects/648782cd11c14c55a2b5138094e6d2e4/users/2e90b7d8bdab40fdb1dddf6db4778308/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:35.869916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf8819-35aa-4ec2-83ff-6af74b9175cf None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:35.870123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf8819-35aa-4ec2-83ff-6af74b9175cf None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:35.870412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf8819-35aa-4ec2-83ff-6af74b9175cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:35.911242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8bcf8819-35aa-4ec2-83ff-6af74b9175cf None None] MFA Rules not processed for user `2e90b7d8bdab40fdb1dddf6db4778308`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:35.912464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8bcf8819-35aa-4ec2-83ff-6af74b9175cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:35.988546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1108/2218] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:36.000604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:36.002334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:36.080440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1d470-035e-4635-9909-c79968766580 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:36.084544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c718a0-1023-4ecb-b845-daec01581039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:36.085918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c718a0-1023-4ecb-b845-daec01581039 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:36.086241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c718a0-1023-4ecb-b845-daec01581039 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:36.086530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c718a0-1023-4ecb-b845-daec01581039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:36.096348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c718a0-1023-4ecb-b845-daec01581039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:36.097795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c718a0-1023-4ecb-b845-daec01581039 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:36.108525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1111/2219] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:37.496197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:37.549142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf8819-35aa-4ec2-83ff-6af74b9175cf None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:37.614646 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf8819-35aa-4ec2-83ff-6af74b9175cf None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:37.620107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:37.621166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:37.621464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:37.621754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] PATH_INFO: `/v3/users/e14b130e3e304044b1c39ba3c85aabfe` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:37.623028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e14b130e3e304044b1c39ba3c85aabfe)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:37.625000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:37.798387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e14b130e3e304044b1c39ba3c85aabfe'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:37.799074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e14b130e3e304044b1c39ba3c85aabfe'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:37.806568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e14b130e3e304044b1c39ba3c85aabfe'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:37.807265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e14b130e3e304044b1c39ba3c85aabfe'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:37.899121 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e14b130e3e304044b1c39ba3c85aabfe'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:37.918633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1109/2220] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:37 2022] DELETE /identity/v3/users/e14b130e3e304044b1c39ba3c85aabfe => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:37.923106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:37.923575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:37.923703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:37.941681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:37.942873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:38.003667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1112/2221] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:38.013646 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:38.015126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:38.077945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595c21-b4e3-42fb-b651-b6a36417a494 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:38.081962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:38.082616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:38.082903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:38.083202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:38.093196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:38.094627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:38.104786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1110/2222] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:38.511897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:38.513297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:38.527082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda54e0-a137-4e24-8693-6c718eb1a051 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:38.536119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:38.537011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:38.537366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:38.537725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] PATH_INFO: `/v3/projects/750892d71cea40bfb560b79c791a107b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:38.540488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750892d71cea40bfb560b79c791a107b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:38.542772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:38.696150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750892d71cea40bfb560b79c791a107b', 'request_id': 'req-65426d5f-166b-4198-ad48-429c119c7941'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:38.703103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750892d71cea40bfb560b79c791a107b', 'request_id': 'req-65426d5f-166b-4198-ad48-429c119c7941'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:38.711989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750892d71cea40bfb560b79c791a107b', 'request_id': 'req-65426d5f-166b-4198-ad48-429c119c7941'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:38.772603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] The token cache is being invalidate because project 750892d71cea40bfb560b79c791a107b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:38.772994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:38.773655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:38.775430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1113/2223] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:38 2022] DELETE /identity/v3/projects/750892d71cea40bfb560b79c791a107b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:38.779905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:38.790578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51f1d9-35d2-4ca1-8916-9924ffe26e75 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:38.849059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:38.849940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:38.850299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:38.850554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] PATH_INFO: `/v3/users/2e90b7d8bdab40fdb1dddf6db4778308` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:38.851801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e90b7d8bdab40fdb1dddf6db4778308)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:38.854194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:38.911636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:39.281947 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e90b7d8bdab40fdb1dddf6db4778308'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:39.282557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e90b7d8bdab40fdb1dddf6db4778308'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:39.290742 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e90b7d8bdab40fdb1dddf6db4778308'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:39.291549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e90b7d8bdab40fdb1dddf6db4778308'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:39.413248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-78e813cf-8911-499f-8b7c-5c7afc78236d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e90b7d8bdab40fdb1dddf6db4778308'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:39.438902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1111/2224] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:38 2022] DELETE /identity/v3/users/2e90b7d8bdab40fdb1dddf6db4778308 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:39.997867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:40.003541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:40.034860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65426d5f-166b-4198-ad48-429c119c7941 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:40.117997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:40.118869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:40.119158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:40.119520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] PATH_INFO: `/v3/projects/648782cd11c14c55a2b5138094e6d2e4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:40.120959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=648782cd11c14c55a2b5138094e6d2e4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:40.123094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:40.150468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:40.363072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '648782cd11c14c55a2b5138094e6d2e4', 'request_id': 'req-43110f2f-2cfe-448a-a432-7aca794e7c88'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:40.370673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '648782cd11c14c55a2b5138094e6d2e4', 'request_id': 'req-43110f2f-2cfe-448a-a432-7aca794e7c88'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:40.379355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '648782cd11c14c55a2b5138094e6d2e4', 'request_id': 'req-43110f2f-2cfe-448a-a432-7aca794e7c88'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:40.708394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] The token cache is being invalidate because project 648782cd11c14c55a2b5138094e6d2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:40.708394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:40.710715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-43110f2f-2cfe-448a-a432-7aca794e7c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:40.713613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1114/2225] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:39 2022] DELETE /identity/v3/projects/648782cd11c14c55a2b5138094e6d2e4 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:40.718667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5de920-3461-4cef-b8b1-fd2af5e375a5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:40.719235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5de920-3461-4cef-b8b1-fd2af5e375a5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:40.720651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5de920-3461-4cef-b8b1-fd2af5e375a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:40.763745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-cf5de920-3461-4cef-b8b1-fd2af5e375a5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:40.765218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cf5de920-3461-4cef-b8b1-fd2af5e375a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:40.831492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1112/2226] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:40.840205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96059-7c60-4e37-8bbe-a55f2030d711 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:40.840601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96059-7c60-4e37-8bbe-a55f2030d711 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:40.840946 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96059-7c60-4e37-8bbe-a55f2030d711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:40.860563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cce96059-7c60-4e37-8bbe-a55f2030d711 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:40.861790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cce96059-7c60-4e37-8bbe-a55f2030d711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:40.869909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1115/2227] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:40.874241 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5de920-3461-4cef-b8b1-fd2af5e375a5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:40.884997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5de920-3461-4cef-b8b1-fd2af5e375a5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:40.890664 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:40.891520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:40.891805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:40.894494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:40.894494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:12:40.894494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:40.895396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:40.909474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1113/2228] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:12:40.918485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce96059-7c60-4e37-8bbe-a55f2030d711 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:40.927503 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce96059-7c60-4e37-8bbe-a55f2030d711 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:40.938198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:40.938968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:40.939738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:40.940606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:40.952941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:40.956991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.103634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1116/2229] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:40 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:41.104437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:41.113989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:41.116576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0215261-7d0b-414f-b05f-4470c96ce3e3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:41.178784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:41.179746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.181885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.181885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.181885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:41.183566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.298909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1114/2230] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:41 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:12:41.303742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:41.334082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c1246-0b46-4edc-a070-aeeca0c4080c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:41.339279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:41.340257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.340545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.340973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.341874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:41.343747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.355755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1117/2231] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:41.359920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:41.370846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16963563-b750-4f62-ba44-54641b9c20a5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:41.375624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbab309-6385-4b9c-ab3d-e7c85d5e9823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:41.376295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbab309-6385-4b9c-ab3d-e7c85d5e9823 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.376580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbab309-6385-4b9c-ab3d-e7c85d5e9823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.376882 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbab309-6385-4b9c-ab3d-e7c85d5e9823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.377646 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbab309-6385-4b9c-ab3d-e7c85d5e9823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:41.379290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbab309-6385-4b9c-ab3d-e7c85d5e9823 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.391454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1115/2232] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:41.398932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:41.409348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5f948-7a83-450b-8bd7-ee1a7a600cdc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:41.415096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:41.416198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.416490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.416769 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] PATH_INFO: `/v3/projects/2354cf0d8555459c8c9cf122c3452758/users/88d1833d76764fd8a206327ce993bcf1/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.434437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2354cf0d8555459c8c9cf122c3452758, user_id=88d1833d76764fd8a206327ce993bcf1, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:41.437201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.533413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1118/2233] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:41 2022] PUT /identity/v3/projects/2354cf0d8555459c8c9cf122c3452758/users/88d1833d76764fd8a206327ce993bcf1/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:41.544242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1f94d-022d-4eb2-bff6-596aaff3d209 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.544836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1f94d-022d-4eb2-bff6-596aaff3d209 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.545322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1f94d-022d-4eb2-bff6-596aaff3d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.579689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-74a1f94d-022d-4eb2-bff6-596aaff3d209 None None] MFA Rules not processed for user `88d1833d76764fd8a206327ce993bcf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:41.581544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-74a1f94d-022d-4eb2-bff6-596aaff3d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:41.662450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1116/2234] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:41.673485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:41.675206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:41.732332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8031-f1ff-4aed-a2dc-9b7898d69db7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:41.735882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6c4a7-251b-4a4d-bd29-644b3ad1b0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:41.736520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c4a7-251b-4a4d-bd29-644b3ad1b0dd service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.736804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c4a7-251b-4a4d-bd29-644b3ad1b0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.737102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c4a7-251b-4a4d-bd29-644b3ad1b0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.745887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6c4a7-251b-4a4d-bd29-644b3ad1b0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:41.747364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6c4a7-251b-4a4d-bd29-644b3ad1b0dd service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.756862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1119/2235] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:41.776858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1f94d-022d-4eb2-bff6-596aaff3d209 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:41.853507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1f94d-022d-4eb2-bff6-596aaff3d209 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:41.858222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:41.859124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.862408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.862408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] PATH_INFO: `/v3/users/88d1833d76764fd8a206327ce993bcf1` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.862408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d1833d76764fd8a206327ce993bcf1)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:41.862876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:41.962772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29b49-7ccd-48fa-b720-ae6d7a18b293 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:41.963065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29b49-7ccd-48fa-b720-ae6d7a18b293 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:41.963378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc29b49-7ccd-48fa-b720-ae6d7a18b293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:41.981806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-afc29b49-7ccd-48fa-b720-ae6d7a18b293 None None] MFA Rules not processed for user `fd3eef3ca9da4302a7c2e52acb8155c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:41.983099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-afc29b49-7ccd-48fa-b720-ae6d7a18b293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:42.037455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1120/2236] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:42.110267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d1833d76764fd8a206327ce993bcf1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:42.111122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d1833d76764fd8a206327ce993bcf1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:42.119412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d1833d76764fd8a206327ce993bcf1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:42.124272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d1833d76764fd8a206327ce993bcf1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:42.224212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d1833d76764fd8a206327ce993bcf1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:42.250820 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1117/2237] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:12:41 2022] DELETE /identity/v3/users/88d1833d76764fd8a206327ce993bcf1 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:42.252989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d6fdf-ca6f-4bd3-9f33-5ac12ba09cde None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:42.254016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d6fdf-ca6f-4bd3-9f33-5ac12ba09cde None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:42.254306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d6fdf-ca6f-4bd3-9f33-5ac12ba09cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:42.280613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f96d6fdf-ca6f-4bd3-9f33-5ac12ba09cde None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:42.282093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f96d6fdf-ca6f-4bd3-9f33-5ac12ba09cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:42.372369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1121/2238] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:42.385244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:42.387136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:42.391044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:42.454218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da603486-ad28-4401-a052-707e9e7a9b50 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:42.458187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:42.458922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:42.459313 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:42.459711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:42.483443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:42.483443 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:42.485438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:42.490968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1118/2239] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:12:43.159356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:43.160088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:43.161189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:43.188127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:43.189989 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:43.200388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:43.205560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:43.210681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b66a0a-5740-407b-a7a3-8a1758add070 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:43.216734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:43.218228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:43.218570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:43.218920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] PATH_INFO: `/v3/projects/2354cf0d8555459c8c9cf122c3452758` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:43.220606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=2354cf0d8555459c8c9cf122c3452758)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:43.223413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:43.290176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1122/2240] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:12:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:43.341813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:43.343284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:43.417204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2218f-a7b3-426f-8727-a182cf4bd815 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:43.434725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2354cf0d8555459c8c9cf122c3452758', 'request_id': 'req-6918091f-5c06-4f81-ab41-3d62441505ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:43.441528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2354cf0d8555459c8c9cf122c3452758', 'request_id': 'req-6918091f-5c06-4f81-ab41-3d62441505ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:43.449541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2354cf0d8555459c8c9cf122c3452758', 'request_id': 'req-6918091f-5c06-4f81-ab41-3d62441505ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:43.482662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b830cd-9b6f-483b-8207-fd37cecfde6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:43.483468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b830cd-9b6f-483b-8207-fd37cecfde6a service swift] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:43.483753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b830cd-9b6f-483b-8207-fd37cecfde6a service swift] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:43.484032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b830cd-9b6f-483b-8207-fd37cecfde6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:43.543325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b830cd-9b6f-483b-8207-fd37cecfde6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:43.544863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b830cd-9b6f-483b-8207-fd37cecfde6a service swift] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:43.554023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1123/2241] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:43.567895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] The token cache is being invalidate because project 2354cf0d8555459c8c9cf122c3452758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:12:43.568197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:43.570423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6918091f-5c06-4f81-ab41-3d62441505ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:12:43.574054 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1119/2242] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:12:43 2022] DELETE /identity/v3/projects/2354cf0d8555459c8c9cf122c3452758 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:12:43.587936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86ab4-03e8-4234-af0f-7bf2a817b4fd None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:43.588236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86ab4-03e8-4234-af0f-7bf2a817b4fd None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:43.588520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86ab4-03e8-4234-af0f-7bf2a817b4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:43.617252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-76e86ab4-03e8-4234-af0f-7bf2a817b4fd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:43.618997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-76e86ab4-03e8-4234-af0f-7bf2a817b4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:43.690783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1124/2243] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:43.700590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c853ec-f32e-4f45-ba09-d64e88adb945 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:43.700898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c853ec-f32e-4f45-ba09-d64e88adb945 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:43.705127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c853ec-f32e-4f45-ba09-d64e88adb945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:43.721488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-50c853ec-f32e-4f45-ba09-d64e88adb945 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:43.724705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-50c853ec-f32e-4f45-ba09-d64e88adb945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:43.731558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1120/2244] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:43.735780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e86ab4-03e8-4234-af0f-7bf2a817b4fd None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:43.746068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e86ab4-03e8-4234-af0f-7bf2a817b4fd None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:43.750768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:43.753574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:43.753574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:43.753574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:43.753574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:12:43.753574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:43.755153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:43.769055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1125/2245] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:12:43.773774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c853ec-f32e-4f45-ba09-d64e88adb945 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:43.785046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c853ec-f32e-4f45-ba09-d64e88adb945 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:43.790941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:43.791720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:43.792070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:43.793844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:43.793844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:43.795315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:43.927093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1121/2246] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:12:43 2022] POST /identity/v3/projects => generated 399 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:43.928308 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:43.947587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac21bfd-ea4f-45df-963a-bb132d903e73 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:44.007523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:44.008214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.008503 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.008784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.009480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:44.011625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:44.146217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1126/2247] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:12:43 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:12:44.149461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:44.162288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694db6-7739-4a91-9120-d50d7f3a5059 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:44.167817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:44.168679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.169016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.169405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.170222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:44.171857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:44.184780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1122/2248] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:44.188608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:44.199712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac3a62-c47b-4288-b7b0-d3194d8c12eb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:44.205418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b861988-067b-4c2c-87df-f6ecb720225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:44.206109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b861988-067b-4c2c-87df-f6ecb720225c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.207582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b861988-067b-4c2c-87df-f6ecb720225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.207876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b861988-067b-4c2c-87df-f6ecb720225c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.208644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b861988-067b-4c2c-87df-f6ecb720225c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:44.210520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b861988-067b-4c2c-87df-f6ecb720225c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:44.221168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1127/2249] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:12:44.226043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:44.236581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c187e2-3a5c-4965-962a-dbcbd7006f18 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:44.246139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:44.246139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.246139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.246139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] PATH_INFO: `/v3/projects/09939f4505b14b178edaf014b973a773/users/2ca43f906fa143848b931abf9dc806ce/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.259171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09939f4505b14b178edaf014b973a773, user_id=2ca43f906fa143848b931abf9dc806ce, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:44.261878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:44.353452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1123/2250] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:12:44 2022] PUT /identity/v3/projects/09939f4505b14b178edaf014b973a773/users/2ca43f906fa143848b931abf9dc806ce/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:12:44.363637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541fc1-a6d0-459d-b768-e5187d89b080 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.364111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541fc1-a6d0-459d-b768-e5187d89b080 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.364347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541fc1-a6d0-459d-b768-e5187d89b080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.421346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ec541fc1-a6d0-459d-b768-e5187d89b080 None None] MFA Rules not processed for user `2ca43f906fa143848b931abf9dc806ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:44.422840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ec541fc1-a6d0-459d-b768-e5187d89b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:44.505001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1128/2251] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:12:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:44.517759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:44.521763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:44.605371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7e7ef-5879-4a64-93ee-730e980db0e3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:44.609187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:44.612099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:44.612889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.617527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.617527 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.622890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:44.624442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:44.633541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d23a1-33c4-4f46-bc8e-137d4826fa23 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.633839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d23a1-33c4-4f46-bc8e-137d4826fa23 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.634360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d23a1-33c4-4f46-bc8e-137d4826fa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.637939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1124/2252] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:12:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:12:44.656879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5d4d23a1-33c4-4f46-bc8e-137d4826fa23 None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:44.658335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5d4d23a1-33c4-4f46-bc8e-137d4826fa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:44.725777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1129/2253] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 11:12:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:12:44.750972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:44.753136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:44.890348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b34f06-0b6c-49d4-9a3b-6672c19f95da service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:44.906471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:44.906471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:44.906471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:44.906471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:44.909581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:44.919506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:44.937738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1125/2254] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:45.836265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:45.939915 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b679c8-596e-4f15-81ce-f2e009aa82de None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:45.940327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b679c8-596e-4f15-81ce-f2e009aa82de None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:45.942125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b679c8-596e-4f15-81ce-f2e009aa82de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:45.968141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-01b679c8-596e-4f15-81ce-f2e009aa82de None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:45.987952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-01b679c8-596e-4f15-81ce-f2e009aa82de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:45.994023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1130/2255] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:12:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:46.009029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:46.020640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ad415-90d3-42b6-8d3e-ea9c58574d7e service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:46.028759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:46.029452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:46.029773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:46.030049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:46.040791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:46.042564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:46.052933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1126/2256] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:47.201511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:47.202012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:47.202105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:47.239159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:12:47.242981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:12:47.275360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1131/2257] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:12:47.276974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:12:47.288285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76a85-8eed-48fc-90fb-6ccbd881a19a service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:12:47.296230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:12:47.296981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:12:47.297852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:12:47.298098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:12:47.307124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:12:47.308908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:12:47.317892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1127/2258] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:12:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:12:48.927747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:49.573581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:50.008482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:51.395960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:12:52.504683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:13:01.213215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=48 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:13:03.064391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=65 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:13:10.735684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:13:10.737085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:10.827418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873b41b-ceb8-4176-bb11-a52403d66ff2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:13:10.832344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:13:10.832987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:13:10.833271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:13:10.833550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:13:10.835985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:13:10.847059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:13:10.848651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:13:10.859073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1132/2259] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:13:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:13:20.855022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=41 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:15:15.734366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:15.755444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bad390-b57c-4fbd-920b-ff2e9a3ac7d8 service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:15.758968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:15.759708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:15.759931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:15.760215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:15.761974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:15.820760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:15.822198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:15.832502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1128/2260] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:15:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:15:19.279636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:19.281024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:19.376125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a03ee-7b4b-44eb-b0ec-54ce246e5632 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:19.382858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:19.383538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:19.383824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:19.384101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:19.385883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:19.442685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:19.444288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:19.453128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1133/2261] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:15:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:15:22.922811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:15:22.931377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a1e92-8779-4c09-afdc-e05e346aa915 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:15:22.934949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:15:22.935619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:15:22.935909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:15:22.936192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:15:22.937995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:15:22.999378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:15:23.000813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:15:23.010080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1129/2262] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:15:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:15:29.448992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=65 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:15:33.005817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=60 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:16:12.940461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:16:12.942031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:16:13.000207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97559e31-389a-413d-9915-7910f3609a1c service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:16:13.003888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:16:13.004536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:16:13.004826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:16:13.005108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:16:13.006854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:16:13.016367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:16:13.017824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:16:13.026683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1134/2263] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:16:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:16:17.794185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:16:17.803594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f9e6d-24e1-46ec-857d-ebe3ec0c79a4 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:16:17.808354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:16:17.810215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:16:17.810215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:16:17.810215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:16:17.812301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:16:17.886891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:16:17.888304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:16:17.898775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1130/2264] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:16:23.022924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=26 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:16:27.893161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=28 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:16:58.718099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:16:58.727387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c19cd-e4a9-4158-bad0-35c0fa987bc8 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:16:58.731591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:16:58.732225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:16:58.732512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:16:58.732784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:16:58.734508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:16:58.794252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:16:58.795717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:16:58.804940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1135/2265] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:16:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:17:08.800531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=28 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:17:10.507491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:17:10.517023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd96c39-074d-4c36-81a0-3ca15cfc3d49 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:17:10.520657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:17:10.521296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:17:10.521582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:17:10.521860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:17:10.523700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:17:10.584175 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:17:10.585660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:17:10.594766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1131/2266] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:17:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:17:20.590554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=28 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:17:44.170247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:17:44.179520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e34b0-ff3e-492a-8c77-2709d1404587 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:17:44.183147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:17:44.183820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:17:44.184107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:17:44.184386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:17:44.192896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:17:44.194331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:17:44.203547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1136/2267] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:17:54.199202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:12.693315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:12.694741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:12.709580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b43f-d99e-4987-ad1a-77de2351573d service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:12.714544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:12.715241 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:12.715532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:12.715812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:12.725023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:12.726596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:12.737265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1132/2268] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:22.472320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:22.481649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5945c-6d3c-4d06-9557-a5788670ba7e service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:22.485843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:22.486479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:22.486761 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:22.487036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:22.488833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:22.503579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:22.505008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:22.515981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1137/2269] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:22.732359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:24.167549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:24.182585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a54de-bdd0-4e36-84df-20ea474e7f08 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:24.189294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d4c6d-1d65-40cd-a6b5-20a1c1dafa15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:24.189817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4c6d-1d65-40cd-a6b5-20a1c1dafa15 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:24.190117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4c6d-1d65-40cd-a6b5-20a1c1dafa15 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:24.190487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4c6d-1d65-40cd-a6b5-20a1c1dafa15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:24.192228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ec2d4c6d-1d65-40cd-a6b5-20a1c1dafa15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:24.209839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d4c6d-1d65-40cd-a6b5-20a1c1dafa15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:24.211682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d4c6d-1d65-40cd-a6b5-20a1c1dafa15 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:24.226687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1133/2270] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:24.899209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:24.909042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beca4a-6b6b-40ae-ae79-eeb15b14cd30 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:24.914011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:24.914672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:24.914961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:24.915266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:24.917122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:24.932275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:24.933768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:24.946091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1138/2271] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:29.348855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d33e5-496c-4367-aad7-e2e7c3f449ff None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:29.349411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d33e5-496c-4367-aad7-e2e7c3f449ff None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:29.349411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d33e5-496c-4367-aad7-e2e7c3f449ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:29.367733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-885d33e5-496c-4367-aad7-e2e7c3f449ff None None] MFA Rules not processed for user `5d079dc5e2b142c7a088648f79d1bb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:18:29.369852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-885d33e5-496c-4367-aad7-e2e7c3f449ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:29.377754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1134/2272] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:18:29.691016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:29.699555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009878bf-16be-4a2b-8fb0-210f6d97104e service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:29.703076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:29.703742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:29.704022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:29.704300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:29.706046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:29.715259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:29.716676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:29.724935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1139/2273] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:32.145673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d33e5-496c-4367-aad7-e2e7c3f449ff None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:32.158177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d33e5-496c-4367-aad7-e2e7c3f449ff None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:32.169041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:32.169041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:32.169041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:32.169041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:32.169460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:32.177200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:32.178760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:32.188458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1135/2274] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:39.721699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=18 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:42.184274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=15 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:44.481404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:44.495842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95b2d-bf85-4b4c-9dbf-cdeca3ecbc7c service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:44.501386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91da8e-53ee-40be-929b-b8bfcc23d4d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:44.502032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91da8e-53ee-40be-929b-b8bfcc23d4d4 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:44.502323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91da8e-53ee-40be-929b-b8bfcc23d4d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:44.502629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91da8e-53ee-40be-929b-b8bfcc23d4d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:44.504926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dc91da8e-53ee-40be-929b-b8bfcc23d4d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:44.517156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91da8e-53ee-40be-929b-b8bfcc23d4d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:44.519227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91da8e-53ee-40be-929b-b8bfcc23d4d4 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:44.530195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1140/2275] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:46.500528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:46.563246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37216365-4169-446c-8f78-d1732a6e4605 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:46.568812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:46.569672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:46.569953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:46.570228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] PATH_INFO: `/v3/users/2ca43f906fa143848b931abf9dc806ce` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:46.571578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca43f906fa143848b931abf9dc806ce)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:46.573557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:46.794337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca43f906fa143848b931abf9dc806ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:46.794997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca43f906fa143848b931abf9dc806ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:46.802704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca43f906fa143848b931abf9dc806ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:46.803376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca43f906fa143848b931abf9dc806ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:46.890063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca43f906fa143848b931abf9dc806ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:46.909543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1136/2276] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:18:46 2022] DELETE /identity/v3/users/2ca43f906fa143848b931abf9dc806ce => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:18:46.914378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:46.914876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:46.915070 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:46.933614 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:18:46.934801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:46.994192 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1141/2277] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:18:47.004468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:47.005851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:47.066957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171126a-5979-4732-9c80-89afe5776248 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:47.070612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:47.071299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:47.071592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:47.071900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:47.082202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:47.083690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:47.093531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1137/2278] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:47.634841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:47.636461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:47.661551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852e716-250f-4a26-bc1d-1873ea167a46 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:47.672368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:47.672368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:47.672368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:47.672368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] PATH_INFO: `/v3/projects/09939f4505b14b178edaf014b973a773` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:47.672790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09939f4505b14b178edaf014b973a773)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:47.672993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:47.776841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09939f4505b14b178edaf014b973a773', 'request_id': 'req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:47.782720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09939f4505b14b178edaf014b973a773', 'request_id': 'req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:47.790302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09939f4505b14b178edaf014b973a773', 'request_id': 'req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:47.878199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] The token cache is being invalidate because project 09939f4505b14b178edaf014b973a773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:18:47.878508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:47.879140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-23e7c08f-098e-417c-9d8c-1ac0067ab1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:47.880741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1142/2279] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:18:47 2022] DELETE /identity/v3/projects/09939f4505b14b178edaf014b973a773 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:18:49.594728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:49.596185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:49.609382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18c32-149c-431d-a955-8b39623a6ea8 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:49.631837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:49.685919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:49.686813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:49.687101 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:49.687421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] PATH_INFO: `/v3/users/27650e513f3c4b8f8d2a4af9308ecb36` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:49.688619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27650e513f3c4b8f8d2a4af9308ecb36)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:49.691305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:49.885506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27650e513f3c4b8f8d2a4af9308ecb36'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:49.886164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27650e513f3c4b8f8d2a4af9308ecb36'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:49.892949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27650e513f3c4b8f8d2a4af9308ecb36'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:49.893920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27650e513f3c4b8f8d2a4af9308ecb36'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:50.002999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27650e513f3c4b8f8d2a4af9308ecb36'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:50.020823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1138/2280] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:18:49 2022] DELETE /identity/v3/users/27650e513f3c4b8f8d2a4af9308ecb36 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:18:50.025242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:50.025749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:50.025851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:50.044538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:18:50.045770 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:50.107017 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1143/2281] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:18:50.116908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:50.118315 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:50.176316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c4ce6-c0de-46ef-8e6f-71dc8a17f5f2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:50.179844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bc946-5f2f-40eb-8da4-a10606a445da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:50.180481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc946-5f2f-40eb-8da4-a10606a445da service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:50.180764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc946-5f2f-40eb-8da4-a10606a445da service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:50.181042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc946-5f2f-40eb-8da4-a10606a445da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:50.189809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bc946-5f2f-40eb-8da4-a10606a445da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:50.191298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bc946-5f2f-40eb-8da4-a10606a445da service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:50.201002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1139/2282] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:50.468176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:50.469613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:50.483146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39744aa6-fee9-48d7-95a5-fedd3189463e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:50.486668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:50.488030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:50.488465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:50.488807 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] PATH_INFO: `/v3/projects/3cdd5d3f0bc2431891a2fc795aeecc08` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:50.490829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cdd5d3f0bc2431891a2fc795aeecc08)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:50.492964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:50.500651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:50.665393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cdd5d3f0bc2431891a2fc795aeecc08', 'request_id': 'req-c573f36e-1be2-42db-8853-a443c0e02b95'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:50.670791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cdd5d3f0bc2431891a2fc795aeecc08', 'request_id': 'req-c573f36e-1be2-42db-8853-a443c0e02b95'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:50.677504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cdd5d3f0bc2431891a2fc795aeecc08', 'request_id': 'req-c573f36e-1be2-42db-8853-a443c0e02b95'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:50.775871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] The token cache is being invalidate because project 3cdd5d3f0bc2431891a2fc795aeecc08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:18:50.776163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:50.776788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-c573f36e-1be2-42db-8853-a443c0e02b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:18:50.778326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1144/2283] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:18:50 2022] DELETE /identity/v3/projects/3cdd5d3f0bc2431891a2fc795aeecc08 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:18:50.784800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432019-f234-4982-a137-5fcb7df8d279 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:50.785270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432019-f234-4982-a137-5fcb7df8d279 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:50.785507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432019-f234-4982-a137-5fcb7df8d279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:50.804927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-31432019-f234-4982-a137-5fcb7df8d279 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:18:50.806137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-31432019-f234-4982-a137-5fcb7df8d279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:50.849861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:50.868603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1140/2284] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:18:50.876475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270a413-b064-41dd-abc0-8392a4cd590d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:50.876846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270a413-b064-41dd-abc0-8392a4cd590d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:50.877132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270a413-b064-41dd-abc0-8392a4cd590d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:50.895276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e270a413-b064-41dd-abc0-8392a4cd590d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:18:50.896475 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e270a413-b064-41dd-abc0-8392a4cd590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:50.904630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1145/2285] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:18:50.909444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31432019-f234-4982-a137-5fcb7df8d279 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:50.919573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31432019-f234-4982-a137-5fcb7df8d279 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:50.924138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:50.936693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:50.936693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:50.936693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:50.936693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:18:50.936693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:50.936693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:50.940199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1141/2286] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:18:50.945260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270a413-b064-41dd-abc0-8392a4cd590d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:50.954943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270a413-b064-41dd-abc0-8392a4cd590d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:50.959733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:50.960418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:50.960704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:50.960985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:50.961719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:50.963666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:51.105697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1146/2287] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:18:50 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:18:51.106842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:51.120944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a2dc1-a4ae-4b41-b1aa-9ce212bc2afc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:51.179477 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:51.180148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:51.180437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:51.180720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:51.181461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:51.183396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:51.308289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1142/2288] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:18:51 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:18:51.312248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:51.322330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae03b56-1d0f-441c-a578-b86e81a40a89 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:51.326892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:51.327574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:51.327858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:51.328135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:51.328858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:51.330456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:51.340259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1147/2289] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:18:51.344202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:51.354700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3be58-61d3-46f5-b55b-2ae47aedf551 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:51.359512 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ac876-9c00-41f2-88b9-744d33d983b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:51.360164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ac876-9c00-41f2-88b9-744d33d983b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:51.360448 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ac876-9c00-41f2-88b9-744d33d983b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:51.360727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ac876-9c00-41f2-88b9-744d33d983b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:51.361484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ac876-9c00-41f2-88b9-744d33d983b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:51.363680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ac876-9c00-41f2-88b9-744d33d983b4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:51.373457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1143/2290] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:18:51.377409 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:51.387424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269b0b9-7895-42bd-bfcb-4063b49875ab admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:51.391962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:51.392833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:51.393121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:51.393433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] PATH_INFO: `/v3/projects/bf219daa10dd46b58efbfa00e2bf51e2/users/49b35583ea7f489fa62dca85b31a1b64/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:51.408233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf219daa10dd46b58efbfa00e2bf51e2, user_id=49b35583ea7f489fa62dca85b31a1b64, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:51.411694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:51.541846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1148/2291] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:18:51 2022] PUT /identity/v3/projects/bf219daa10dd46b58efbfa00e2bf51e2/users/49b35583ea7f489fa62dca85b31a1b64/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:18:51.547041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d5d7-ef1e-4850-89b4-44869cccbd23 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:51.547369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d5d7-ef1e-4850-89b4-44869cccbd23 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:51.547674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73d5d7-ef1e-4850-89b4-44869cccbd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:51.565678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3b73d5d7-ef1e-4850-89b4-44869cccbd23 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:18:51.566952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3b73d5d7-ef1e-4850-89b4-44869cccbd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:51.625823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1144/2292] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:18:51.635379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:51.637082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:51.694513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d09a-af5a-4e0c-b462-1bf76440823a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:51.698327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:51.698974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:51.699294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:51.699575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:51.709630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:51.711071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:51.720910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1149/2293] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:18:57.199777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b62e65-8e77-47b9-8b0f-8e32b9788bd4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:57.200295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b62e65-8e77-47b9-8b0f-8e32b9788bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:57.200409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b62e65-8e77-47b9-8b0f-8e32b9788bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:57.229159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-86b62e65-8e77-47b9-8b0f-8e32b9788bd4 None None] MFA Rules not processed for user `49b35583ea7f489fa62dca85b31a1b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:18:57.230376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-86b62e65-8e77-47b9-8b0f-8e32b9788bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:18:57.281258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:57.308440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1145/2294] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:18:57.312973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:57.324739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27476bd0-f386-4cf0-a28d-23af3441d7e4 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:57.327907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:57.328631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:57.328863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:57.329143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:57.329838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:57.332142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:57.463570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1150/2295] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:18:57 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:18:57.467868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b62e65-8e77-47b9-8b0f-8e32b9788bd4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:57.480958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b62e65-8e77-47b9-8b0f-8e32b9788bd4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:57.553900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:57.554749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:57.555039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:57.555353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:57.556051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:57.558029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:57.645270 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1146/2296] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:18:57 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:18:57.649218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:57.655380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:18:57.661775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0012a-bf85-42c2-85ba-dc8d8c9591da admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:57.666694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:57.667412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:57.667707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:57.667992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:57.668736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:57.670452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:57.681481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1151/2297] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:18:57.685750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:57.696332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe14c32-e4d9-41cb-95b0-de4d240d831f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:57.701912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:57.702814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:57.703103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:57.703421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] PATH_INFO: `/v3/projects/db4f40be37054b608841b8e926f0b553/users/556daa515e71430badb513338c4d7d3e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:57.720888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4f40be37054b608841b8e926f0b553, user_id=556daa515e71430badb513338c4d7d3e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:57.724253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:57.833429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1147/2298] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:18:57 2022] PUT /identity/v3/projects/db4f40be37054b608841b8e926f0b553/users/556daa515e71430badb513338c4d7d3e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:18:57.837340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:57.848175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be3fa-b43e-42eb-99f9-e2a42aba9db1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:57.907531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492a18c-ecab-4133-b92d-eb543b8651b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:57.908221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a18c-ecab-4133-b92d-eb543b8651b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:57.908507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a18c-ecab-4133-b92d-eb543b8651b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:57.909394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a18c-ecab-4133-b92d-eb543b8651b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:57.910149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492a18c-ecab-4133-b92d-eb543b8651b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:57.911793 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492a18c-ecab-4133-b92d-eb543b8651b2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:57.922321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1152/2299] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:18:57.926828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:18:57.938507 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98638e-f207-469f-acb6-cacadbf44cae admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:18:57.944075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:18:57.945736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:18:57.945736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:18:57.945736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] PATH_INFO: `/v3/projects/db4f40be37054b608841b8e926f0b553/users/556daa515e71430badb513338c4d7d3e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:18:57.947570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4f40be37054b608841b8e926f0b553, user_id=556daa515e71430badb513338c4d7d3e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:18:57.950232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:18:57.961711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1148/2300] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:18:57 2022] PUT /identity/v3/projects/db4f40be37054b608841b8e926f0b553/users/556daa515e71430badb513338c4d7d3e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:19:03.684885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961c33-7894-4c41-baf6-9797e25604b7 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:03.685917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961c33-7894-4c41-baf6-9797e25604b7 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:03.686497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961c33-7894-4c41-baf6-9797e25604b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:03.723217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3f961c33-7894-4c41-baf6-9797e25604b7 None None] MFA Rules not processed for user `556daa515e71430badb513338c4d7d3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:19:03.724609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3f961c33-7894-4c41-baf6-9797e25604b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:03.802605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1153/2301] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:19:03.814309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:19:03.816239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:03.912212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731b44-423b-4ca1-a50c-571c270a853a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:19:03.917453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:19:03.918401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:03.919531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:03.920020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:03.950119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:19:03.984570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:19:03.986157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:19:03.997349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1149/2302] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:19:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:19:04.237895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388e2b8-a13e-45f7-a121-36c6c2bf4f19 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:04.238210 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388e2b8-a13e-45f7-a121-36c6c2bf4f19 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:04.238547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388e2b8-a13e-45f7-a121-36c6c2bf4f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:04.287151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7388e2b8-a13e-45f7-a121-36c6c2bf4f19 None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:19:04.288387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7388e2b8-a13e-45f7-a121-36c6c2bf4f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:04.371826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:19:04.390366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1154/2303] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:19:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:19:04.550398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:19:04.551843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:04.647901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e710c9-2a99-4f84-a744-662a09aaa564 service glance] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:19:04.653003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:19:04.653927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:04.654195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:04.654480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:04.663307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:19:04.664923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:19:04.673102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1150/2304] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:19:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:19:14.408452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:19:14.669779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=56 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:19:16.156515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08af513-9a02-4ae5-8e80-e3868359e1da None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:16.157092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08af513-9a02-4ae5-8e80-e3868359e1da None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:16.157092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08af513-9a02-4ae5-8e80-e3868359e1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:16.174481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b08af513-9a02-4ae5-8e80-e3868359e1da None None] MFA Rules not processed for user `49b35583ea7f489fa62dca85b31a1b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:19:16.175907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b08af513-9a02-4ae5-8e80-e3868359e1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:16.182826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1155/2305] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:19:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:19:16.445653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:19:16.447100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:16.503274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04238c44-42e7-42fe-813f-1656cc7f86a8 service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:19:16.507416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:19:16.508057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:16.508345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:16.508626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:16.510392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:16.519694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:19:16.521127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:19:16.529231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1151/2306] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:19:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:19:19.907458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:19.907950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:19.908039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:19.926892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:19:19.928145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:19:19.935406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1156/2307] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 11:19:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:19:19.948246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:19:19.961298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdcaba-42d8-45a1-9dd4-e41727ffeab3 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:19:19.966301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:19:19.966940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:19:19.967275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:19:19.967561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:19:19.975756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:19:19.977339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:19:19.985662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1152/2308] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:19:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:19:29.921104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:19:29.982119 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=32 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:15.362856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:15.364324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:15.434339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750d745-0012-43ad-8638-fb9fcd7bcce6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:15.438019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:15.438672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:15.438962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:15.439293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:15.441077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:15.521087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:15.522554 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:15.541040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1157/2309] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:20:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:20:19.496550 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:19.498047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:19.574090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cff53-24b2-4101-980c-cb936475e697 service swift] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:19.579474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:19.580123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:19.580409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:19.580687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:19.582420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:19.663453 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:19.665093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:19.684055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1153/2310] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:20:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:20:22.600728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:22.609234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08613572-de6e-4c3e-9cd1-3b1731128171 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:22.612887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7718f-c686-4bb1-b3eb-ac5c6db4c850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:22.613523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7718f-c686-4bb1-b3eb-ac5c6db4c850 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:22.613809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7718f-c686-4bb1-b3eb-ac5c6db4c850 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:22.614089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7718f-c686-4bb1-b3eb-ac5c6db4c850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:22.616658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8fe7718f-c686-4bb1-b3eb-ac5c6db4c850 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:22.694042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:22.698009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7718f-c686-4bb1-b3eb-ac5c6db4c850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:22.699544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7718f-c686-4bb1-b3eb-ac5c6db4c850 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:22.717865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1158/2311] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:20:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:20:29.681451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=69 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:32.721692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:43.027207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:43.036952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e8dd9-a367-4753-b661-64e3587e2382 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:43.040625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ede2f-3808-4d76-8bb4-813ea6e2c887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:43.041273 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ede2f-3808-4d76-8bb4-813ea6e2c887 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:43.041563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ede2f-3808-4d76-8bb4-813ea6e2c887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:43.041845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ede2f-3808-4d76-8bb4-813ea6e2c887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:43.043810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0d1ede2f-3808-4d76-8bb4-813ea6e2c887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:43.109387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ede2f-3808-4d76-8bb4-813ea6e2c887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:43.110834 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ede2f-3808-4d76-8bb4-813ea6e2c887 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:43.122013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1154/2312] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:20:43.694297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cad87d-f4d0-47fa-af55-b61f2523c60a None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:43.694816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cad87d-f4d0-47fa-af55-b61f2523c60a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:43.694927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cad87d-f4d0-47fa-af55-b61f2523c60a None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:43.696604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1159/2313] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:20:43.708381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:43.708809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:43.709090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:43.747773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] MFA Rules not processed for user `80003dcc49ef47c8a08887462d102b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:20:43.749012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:43.812293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1155/2314] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:20:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:20:43.822913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cad87d-f4d0-47fa-af55-b61f2523c60a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:43.824753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-27cad87d-f4d0-47fa-af55-b61f2523c60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:43.841508 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cad87d-f4d0-47fa-af55-b61f2523c60a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:43.846589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:43.847254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:43.847540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:43.847818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:43.858318 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:43.859935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:43.874088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1160/2315] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:20:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:20:43.888147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:43.889635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:43.918355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd69df5-f993-432a-94ec-5f2c9867a818 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:43.926459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:43.927514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:43.927884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:43.928440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] PATH_INFO: `/v3/users/aa342d57ade24c91aa33a9c71447b846` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:43.932857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa342d57ade24c91aa33a9c71447b846)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:43.934856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:44.127612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa342d57ade24c91aa33a9c71447b846'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.128745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa342d57ade24c91aa33a9c71447b846'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.136926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa342d57ade24c91aa33a9c71447b846'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.137582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa342d57ade24c91aa33a9c71447b846'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.192405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-337927a2-f63a-4447-bb6b-0b7638400639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa342d57ade24c91aa33a9c71447b846'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.200391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:44.211015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1156/2316] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:20:43 2022] DELETE /identity/v3/users/aa342d57ade24c91aa33a9c71447b846 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:20:44.449892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:44.460935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2408b-32c5-433d-8b70-4173da267f8e service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:44.519786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:44.520648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:44.520930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:44.521230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] PATH_INFO: `/v3/projects/e5f021fe083b45c28ad560c481f307cf` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:44.522396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f021fe083b45c28ad560c481f307cf)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:44.524346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:44.624922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f021fe083b45c28ad560c481f307cf', 'request_id': 'req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.630411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f021fe083b45c28ad560c481f307cf', 'request_id': 'req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.637442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f021fe083b45c28ad560c481f307cf', 'request_id': 'req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.713357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] The token cache is being invalidate because project e5f021fe083b45c28ad560c481f307cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:20:44.714086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.714390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b00a4a99-dc73-4dc2-9bef-b51778b4f33b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:20:44.716062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1161/2317] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:20:44 2022] DELETE /identity/v3/projects/e5f021fe083b45c28ad560c481f307cf => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:20:44.722850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11a64a-0d47-4a36-9973-029c33349dc5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:44.723147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11a64a-0d47-4a36-9973-029c33349dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:44.723464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11a64a-0d47-4a36-9973-029c33349dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:44.744733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4e11a64a-0d47-4a36-9973-029c33349dc5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:20:44.746783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4e11a64a-0d47-4a36-9973-029c33349dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:44.808002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1157/2318] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:20:44.815718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc7c43-7a26-450f-801b-d3db6b0a3948 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:44.816011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc7c43-7a26-450f-801b-d3db6b0a3948 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:44.816289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc7c43-7a26-450f-801b-d3db6b0a3948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:44.834340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8cbc7c43-7a26-450f-801b-d3db6b0a3948 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:20:44.835571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8cbc7c43-7a26-450f-801b-d3db6b0a3948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:44.843973 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1162/2319] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:20:44.848053 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11a64a-0d47-4a36-9973-029c33349dc5 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:44.858281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11a64a-0d47-4a36-9973-029c33349dc5 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:44.862819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:44.863505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:44.863793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:44.864076 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:44.864833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:20:44.865144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:44.866767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:44.879230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1158/2320] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:20:44.883526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc7c43-7a26-450f-801b-d3db6b0a3948 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:44.893975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc7c43-7a26-450f-801b-d3db6b0a3948 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:44.899012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:44.899721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:44.900381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:44.900703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:44.902238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:44.904324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:44.994702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1163/2321] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:20:44 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:20:45.000204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:45.010416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcb765-6a3f-4129-a3e3-5ea0fce35ee6 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:45.067299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:45.067973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:45.068260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:45.068585 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:45.069275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:45.071243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:45.173400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1159/2322] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:20:44 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:20:45.177214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:45.187809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fe435-52c1-4ebf-bd93-bb78d9e9a51d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:45.192422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:45.193076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:45.193361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:45.193638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:45.194408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:45.196053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:45.207188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1164/2323] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:20:45.211047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:45.221734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd5ba1-8864-47ad-8ebc-bac0ccdf2bbd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:45.226523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d231b-25f4-485a-9ec8-8387e1171bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:45.227202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d231b-25f4-485a-9ec8-8387e1171bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:45.227491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d231b-25f4-485a-9ec8-8387e1171bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:45.227774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d231b-25f4-485a-9ec8-8387e1171bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:45.228504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d231b-25f4-485a-9ec8-8387e1171bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:45.230216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d231b-25f4-485a-9ec8-8387e1171bb8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:45.239943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1160/2324] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:20:45.243802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:45.254212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffe7f5-cc1d-47ce-a7b5-66045bf1461c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:45.258845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:45.259747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:45.260036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:45.260317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] PATH_INFO: `/v3/projects/bb07d6b829814af5af6afa53a1e4a3ad/users/6ae035f9078540adabf1e7578ef06a3d/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:45.275636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb07d6b829814af5af6afa53a1e4a3ad, user_id=6ae035f9078540adabf1e7578ef06a3d, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:45.278316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:45.413645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1165/2325] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:20:45 2022] PUT /identity/v3/projects/bb07d6b829814af5af6afa53a1e4a3ad/users/6ae035f9078540adabf1e7578ef06a3d/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:20:45.424075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:45.424360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:45.424646 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:45.452167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] MFA Rules not processed for user `6ae035f9078540adabf1e7578ef06a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:20:45.453383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:45.497382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:45.518574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1161/2326] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:20:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:20:45.531420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:20:45.533122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:20:45.547901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:45.592302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94c1fd-64e5-450d-8d47-83a1b6ebd363 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:20:45.596100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:20:45.596776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:20:45.597068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:20:45.597353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:20:45.606469 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:20:45.607962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:20:45.617133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1166/2327] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:20:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:20:55.612604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:20:55.666619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=20 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:21:18.032085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:21:18.033577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:21:18.091603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4f136-0eab-42cf-b86f-660d7aa89754 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:21:18.095388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:21:18.096040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:21:18.096331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:21:18.097101 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:21:18.098931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:21:18.181327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:21:18.182772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:21:18.200694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1162/2328] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:21:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:21:18.386402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:21:18.387932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:21:18.444862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f2ff8-00b5-4b2b-b2e0-15ee9ea276ef service glance] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:21:18.448530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:21:18.449177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:21:18.449461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:21:18.449739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:21:18.451531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:21:18.461679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:21:18.463104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:21:18.470944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1167/2329] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:21:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:21:28.198061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=61 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:21:28.467641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=26 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:23:19.664702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:23:19.666555 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:23:19.815875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971932e6-1980-4bdf-87b3-64852d0df5d1 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:23:19.820926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:23:19.821553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:23:19.821839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:23:19.822115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:23:19.823902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:23:19.835434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:23:19.837049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:23:19.847569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1163/2330] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:23:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:23:29.842543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=63 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:04.939135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:04.940640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:04.981815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4694e05-2e8c-4846-861f-dd66284cd49d service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:04.985380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:04.986044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:04.986326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:04.986601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:04.988877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:05.047122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:05.048579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:05.058092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1168/2331] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:24:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:24:15.054012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=41 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:36.923254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:36.942250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714951c-3ab3-47ec-9476-2eddf8814382 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:36.946649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:36.947250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:36.947545 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:36.947818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:36.949548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:37.029346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:37.030768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:37.041738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1164/2332] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:24:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:24:38.215150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:24:38.224371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de04cf4-ef1b-49df-8f92-2296d18f1379 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:24:38.228717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:24:38.229365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:24:38.229651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:24:38.229921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:24:38.231728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:24:38.245085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:24:38.246507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:24:38.257444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1169/2333] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:24:47.036689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=41 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:24:48.252542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:16.435151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:16.453889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f439d-5fa4-4bbc-b4da-4593f0748f89 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:16.457494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:16.458131 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:16.458418 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:16.458700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:16.460492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:16.519492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:16.520928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:16.531839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1165/2334] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:25:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:25:19.890784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:19.900279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70acab2-fd65-4ed9-a27f-d1a8d9e6d707 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:19.904722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:19.905400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:19.905684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:19.905961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:19.907796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:19.968464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:19.969896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:19.978738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1170/2335] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:25:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:25:22.619570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:22.628919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6977-9f5c-46ab-80af-1c49c2dc6bd3 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:22.632497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:22.633136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:22.633421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:22.633699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:22.635477 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:22.695991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:22.697435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:22.706871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1166/2336] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:25:27.542696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:27.545004 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:27.602703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc8751-2106-4d31-8545-7bf1c19eca93 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:27.609413 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:27.610157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:27.611652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:27.611924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:27.614358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:25:27.629959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:27.631649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:27.660317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1171/2337] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:25:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:25:32.702358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=60 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:37.645933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=43 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:25:44.787975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:25:44.797320 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b06c05-1acc-48d2-a51a-1c665569921e service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:25:44.800935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:25:44.801569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:25:44.801853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:25:44.802134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:25:44.810399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:25:44.811794 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:25:44.820439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1167/2338] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:25:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:25:54.821174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:17.650688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:17.652127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:26:17.666711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbb742-3659-44cd-9fc5-a30cb6acc71d service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:17.672291 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:17.672927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:17.673215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:17.673496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:17.681246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:17.683489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:17.692679 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1172/2339] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:26:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:26:22.367506 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:26:22.378263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8be5ba-d121-43a0-a021-ffd6c4a1f5bb service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:26:22.382104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:26:22.382746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:26:22.383030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:26:22.383349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:26:22.391283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:26:22.392786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:26:22.401003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1168/2340] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:26:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:26:27.688599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:26:32.397434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:27:57.675169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:27:57.676939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:57.701838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879bd4c-f235-4ce3-b814-20f41abc1f21 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:27:57.705396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:27:57.706029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:27:57.706312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:27:57.706615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:27:57.709100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:27:57.792692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:27:57.794129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:27:57.813444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1173/2341] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:01.378759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:01.394022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cba4fd-97e6-4daa-9431-1c6d38b15da3 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:01.400353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:01.401039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:01.401469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:01.401756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:01.403685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:01.419730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:01.421341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:01.433403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1169/2342] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:28:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:01.510096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:01.511717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:01.526493 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994968e8-34f1-43d0-b884-1656bcc1819c service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:01.531563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:01.532444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:01.532735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:01.533020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] PATH_INFO: `/v3/users/0fbb20d3c4bf4ed09a4e8c01ca10a927` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:01.534251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbb20d3c4bf4ed09a4e8c01ca10a927)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:01.537489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:01.749039 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbb20d3c4bf4ed09a4e8c01ca10a927'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:01.758207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbb20d3c4bf4ed09a4e8c01ca10a927'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:01.759298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbb20d3c4bf4ed09a4e8c01ca10a927'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:01.763911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbb20d3c4bf4ed09a4e8c01ca10a927'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:01.778836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f052c263-fa07-40b1-9216-386f266e454f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbb20d3c4bf4ed09a4e8c01ca10a927'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:01.848541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1174/2343] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:28:01 2022] DELETE /identity/v3/users/0fbb20d3c4bf4ed09a4e8c01ca10a927 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:28:02.106249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:02.117782 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cfb70-ba68-4d6a-ad4b-fa0a64bb16e4 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:02.177807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:02.178663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.178950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.179261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] PATH_INFO: `/v3/projects/3622ea9c334d40b08ca745f89be6b988` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.180449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3622ea9c334d40b08ca745f89be6b988)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:02.182342 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:02.259533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3622ea9c334d40b08ca745f89be6b988', 'request_id': 'req-89a5b4f3-74ff-4678-bdd8-c13840029301'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:02.264963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3622ea9c334d40b08ca745f89be6b988', 'request_id': 'req-89a5b4f3-74ff-4678-bdd8-c13840029301'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:02.271764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3622ea9c334d40b08ca745f89be6b988', 'request_id': 'req-89a5b4f3-74ff-4678-bdd8-c13840029301'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:02.325876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] The token cache is being invalidate because project 3622ea9c334d40b08ca745f89be6b988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:28:02.326174 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:02.326793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-89a5b4f3-74ff-4678-bdd8-c13840029301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:28:02.328376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1170/2344] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:28:02 2022] DELETE /identity/v3/projects/3622ea9c334d40b08ca745f89be6b988 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:28:02.334649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73245acb-3713-400b-9565-e6ab6250b0cb None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.334948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73245acb-3713-400b-9565-e6ab6250b0cb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.335262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73245acb-3713-400b-9565-e6ab6250b0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.371977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:28:02.392603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-73245acb-3713-400b-9565-e6ab6250b0cb None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:28:02.393806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-73245acb-3713-400b-9565-e6ab6250b0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:02.454631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1175/2345] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:28:02.462818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55e4d2-2597-4251-aa68-2bacb2ec9e10 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.463383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55e4d2-2597-4251-aa68-2bacb2ec9e10 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.463500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55e4d2-2597-4251-aa68-2bacb2ec9e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.482658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3c55e4d2-2597-4251-aa68-2bacb2ec9e10 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:28:02.483913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3c55e4d2-2597-4251-aa68-2bacb2ec9e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:02.492352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1171/2346] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:28:02.496459 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73245acb-3713-400b-9565-e6ab6250b0cb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:02.507232 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73245acb-3713-400b-9565-e6ab6250b0cb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:02.512008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:02.512661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.512945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.513225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.513955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:28:02.514265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:02.515918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:02.527521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1176/2347] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:28:02.531835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55e4d2-2597-4251-aa68-2bacb2ec9e10 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:02.541117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55e4d2-2597-4251-aa68-2bacb2ec9e10 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:02.545753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:02.546399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.546684 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.546959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.547689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:02.549597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:02.660849 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1172/2348] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:28:02 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:28:02.665417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:02.676639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cb91b-f7f0-4728-822d-a06418c185ba admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:02.737723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:02.738927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.739349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.739561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.740398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:02.742567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:02.814215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1177/2349] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:28:02 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:28:02.820466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:02.835006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c20a57-60a9-4d88-8681-4dcf0885344c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:02.840060 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:02.840766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.841136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.841489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.842337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:02.844059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:02.858356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1173/2350] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:28:02.862214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:02.873142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a973f1-2fdd-400b-aa2e-40a24f6edf9d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:02.877929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c1090-6c03-4182-8325-43bbbc23ed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:02.878626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1090-6c03-4182-8325-43bbbc23ed7a admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.878923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1090-6c03-4182-8325-43bbbc23ed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.879219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c1090-6c03-4182-8325-43bbbc23ed7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.880020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c1090-6c03-4182-8325-43bbbc23ed7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:02.881592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c1090-6c03-4182-8325-43bbbc23ed7a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:02.895040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1178/2351] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:28:02.897312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:02.909406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf965fea-3433-4e8f-bcfa-a9d6a1bb5c6f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:02.918878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:02.919878 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:02.920194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:02.920481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] PATH_INFO: `/v3/projects/cd28037704a444e6af8977fd3fa89470/users/2d2a08af4880453287c2f6b968bd8a27/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:02.936447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd28037704a444e6af8977fd3fa89470, user_id=2d2a08af4880453287c2f6b968bd8a27, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:02.939138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:03.001575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1174/2352] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:28:02 2022] PUT /identity/v3/projects/cd28037704a444e6af8977fd3fa89470/users/2d2a08af4880453287c2f6b968bd8a27/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:28:03.006980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a6179-512f-4f9a-b172-2607e19d35c0 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:03.007440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a6179-512f-4f9a-b172-2607e19d35c0 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:03.008051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a6179-512f-4f9a-b172-2607e19d35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:03.026125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6f3a6179-512f-4f9a-b172-2607e19d35c0 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:28:03.027383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6f3a6179-512f-4f9a-b172-2607e19d35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:03.088146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1179/2353] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:28:03.098144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:03.100021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:03.114431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:28:03.158244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc108f-a694-498d-b67d-f1757051466b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:03.162355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:03.166080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:03.166080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:03.166080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:03.172638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:03.174105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:03.184006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1175/2354] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:08.385639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8d73-81b3-417b-9e14-6f0b980106ba None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:08.386234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8d73-81b3-417b-9e14-6f0b980106ba None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:08.386234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d8d73-81b3-417b-9e14-6f0b980106ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:08.393186 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:28:08.415762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f26d8d73-81b3-417b-9e14-6f0b980106ba None None] MFA Rules not processed for user `2d2a08af4880453287c2f6b968bd8a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:28:08.448675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f26d8d73-81b3-417b-9e14-6f0b980106ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:08.488480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1180/2355] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:28:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:28:08.502279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:08.503784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:08.581238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0953a-fffa-40f1-8b17-0a7d5b6ab4cd service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:08.587380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:08.588038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:08.588332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:08.588611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:08.597895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:08.599543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:08.611442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1176/2356] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:28:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:12.009095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:12.009636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:12.009757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:12.029964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] MFA Rules not processed for user `2d2a08af4880453287c2f6b968bd8a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:28:12.031277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:12.039654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1181/2357] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:28:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:28:12.375581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:12.377125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:12.390000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcd901-c98b-4436-b095-ffd03b9e3bcf service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:12.393522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:12.394871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:12.395153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:12.395455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:12.397231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:12.406628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:12.408092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:12.416260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1177/2358] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:28:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:12.833436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:12.834918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:12.912975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06352379-89ec-422d-99f9-74fbcd26e255 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:12.918363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:12.919270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:12.919685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:12.920041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:12.922172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:12.935822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:12.937434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:12.948510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1182/2359] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:28:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:21.480720 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:21.494115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026674f-5f33-45fd-bedb-7a3e26433341 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:21.499688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:21.500331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:21.500617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:21.500898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:21.502644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:21.515795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:21.517361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:21.528967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1178/2360] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:28:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:22.944087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=66 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:28:25.752296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:28:25.763373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036906d-e47a-4403-874d-b391f0b07462 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:28:25.768875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbb186-3dc8-4150-b11c-e660a3eb83ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:28:25.770288 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb186-3dc8-4150-b11c-e660a3eb83ab service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:28:25.770576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb186-3dc8-4150-b11c-e660a3eb83ab service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:28:25.770853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb186-3dc8-4150-b11c-e660a3eb83ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:28:25.772663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6bcbb186-3dc8-4150-b11c-e660a3eb83ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:28:25.785671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbb186-3dc8-4150-b11c-e660a3eb83ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:28:25.787381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbb186-3dc8-4150-b11c-e660a3eb83ab service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:28:25.797937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1183/2361] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:28:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:28:31.524581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=66 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:28:35.794169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:29:36.503928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:29:36.514415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96b810-ef90-42ad-9960-28d9dcbe2bca service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:29:36.518239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:29:36.518876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:29:36.519163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:29:36.519466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:29:36.521244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:29:36.535374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:29:36.537012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:29:36.548392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1179/2362] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:29:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:29:46.543495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:30:09.673071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:30:09.673608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:30:09.673745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:30:09.691898 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] MFA Rules not processed for user `2d2a08af4880453287c2f6b968bd8a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:30:09.693147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:30:09.700236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1184/2363] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:30:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:30:18.984944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:30:18.996958 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad449c2-248e-4b87-85b9-0e22679c4ae1 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:30:19.002627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:30:19.003391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:30:19.003676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:30:19.003952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:30:19.005704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:30:19.063845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:30:19.065427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:30:19.074661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1180/2364] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:30:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:30:19.686349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:30:29.070408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=28 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:30:47.506095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:30:47.508075 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:30:47.575550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb98e6e-d3b3-498a-8d5d-b332e86b46ae None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:30:47.579095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:30:47.579838 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:30:47.580127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:30:47.580411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:30:47.582184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:30:47.660680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:30:47.662104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:30:47.681762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1185/2365] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:30:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:30:47.724397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:30:47.734100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a050bb-6e30-4cca-94d4-a6778ea7222d service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:30:47.737645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:30:47.738278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:30:47.738565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:30:47.738841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:30:47.740615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:30:47.822347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:30:47.823845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:30:47.842746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1181/2366] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:30:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:30:47.885790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:30:47.894157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6949c-0ee3-43e0-9f97-7b5ede9c199e service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:30:47.898304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:30:47.898942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:30:47.899331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:30:47.899618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:30:47.901403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:30:47.979219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:30:47.982799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:30:47.984278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:30:48.002257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1186/2367] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:30:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:30:57.840069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=41 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:30:58.046898 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:31:28.342736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:31:28.352004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4b99-8897-45c9-9938-225722eea621 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:31:28.356196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:31:28.356833 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:31:28.357120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:31:28.357397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:31:28.359160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:31:28.368470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:31:28.369894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:31:28.378102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1182/2368] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:31:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:31:38.374572 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:32:02.527983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:02.540931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c0278-5838-4340-81db-ad6fe62a1896 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:02.547113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:02.547850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:02.548154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:02.548450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:02.550253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:02.568349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:02.569954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:02.582712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1187/2369] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:32:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:32:08.327491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:08.343554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0384d-ed7a-4b7c-9d9d-f2374bf83cf6 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:08.350038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:08.350880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:08.351298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:08.351675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:08.354614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:08.367074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:08.368889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:08.380691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1183/2370] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:32:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:32:08.465903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:08.478677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc16f94-ebbd-4047-93ec-4e0854d9d052 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:08.482610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:08.483515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:08.483801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:08.484180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] PATH_INFO: `/v3/users/2d2a08af4880453287c2f6b968bd8a27` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:08.485376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2a08af4880453287c2f6b968bd8a27)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:08.487353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:08.642530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2a08af4880453287c2f6b968bd8a27'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:08.649590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2a08af4880453287c2f6b968bd8a27'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:08.650300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2a08af4880453287c2f6b968bd8a27'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:08.650966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2a08af4880453287c2f6b968bd8a27'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:08.662717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d869d7ae-a83a-4d0c-84b4-e56f4d97933e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2a08af4880453287c2f6b968bd8a27'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:08.742132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1188/2371] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:32:08 2022] DELETE /identity/v3/users/2d2a08af4880453287c2f6b968bd8a27 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:32:08.978633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:08.991143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f66f1-53b5-47e7-aaa5-cb6b19e41fb2 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:09.050243 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:09.051125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.051422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.051700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] PATH_INFO: `/v3/projects/cd28037704a444e6af8977fd3fa89470` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.052883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd28037704a444e6af8977fd3fa89470)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:09.054798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:09.165759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd28037704a444e6af8977fd3fa89470', 'request_id': 'req-dfc0b855-7b1a-46de-b97d-f7e323da8e08'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:09.171612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd28037704a444e6af8977fd3fa89470', 'request_id': 'req-dfc0b855-7b1a-46de-b97d-f7e323da8e08'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:09.177785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd28037704a444e6af8977fd3fa89470', 'request_id': 'req-dfc0b855-7b1a-46de-b97d-f7e323da8e08'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:09.262115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] The token cache is being invalidate because project cd28037704a444e6af8977fd3fa89470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:32:09.262411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:09.263038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dfc0b855-7b1a-46de-b97d-f7e323da8e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:32:09.264651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1184/2372] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:32:08 2022] DELETE /identity/v3/projects/cd28037704a444e6af8977fd3fa89470 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:32:09.269494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323fa4a-b58f-41f5-9f42-edc08784db20 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.269789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323fa4a-b58f-41f5-9f42-edc08784db20 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.270073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323fa4a-b58f-41f5-9f42-edc08784db20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.289600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3323fa4a-b58f-41f5-9f42-edc08784db20 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:32:09.290811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3323fa4a-b58f-41f5-9f42-edc08784db20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:09.348537 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1189/2373] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:32:09.357225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7ea5-46cc-4957-97d0-3cdc4996ee09 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.357733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7ea5-46cc-4957-97d0-3cdc4996ee09 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.357843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7ea5-46cc-4957-97d0-3cdc4996ee09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.376262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7aaf7ea5-46cc-4957-97d0-3cdc4996ee09 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:32:09.377482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7aaf7ea5-46cc-4957-97d0-3cdc4996ee09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:09.385301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1185/2374] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:32:09.389369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323fa4a-b58f-41f5-9f42-edc08784db20 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:09.399403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323fa4a-b58f-41f5-9f42-edc08784db20 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:09.404488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:09.405157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.405776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.406069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.406801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:32:09.407114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:09.408755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:09.421479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1190/2375] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:32:09.425543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf7ea5-46cc-4957-97d0-3cdc4996ee09 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:09.435104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf7ea5-46cc-4957-97d0-3cdc4996ee09 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:09.440079 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:09.440498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.440785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.441061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.441754 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:09.443677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:09.533702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1186/2376] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:32:09 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:32:09.537728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:09.549177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c63adf-7aba-40c4-992c-53aff412fbaa admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:09.592603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:32:09.605735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:09.606468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.606757 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.607041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.607800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:09.609847 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:09.714788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1191/2377] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:32:09 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:32:09.718899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:09.729285 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948756e3-b9ca-4f9f-9590-c0dd1eaff586 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:09.733922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:09.734579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.734865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.735146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.735916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:09.737511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:09.747711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1187/2378] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:32:09.755871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:09.766463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9728c-2ec7-44c5-a1b5-e4820bbc981e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:09.771215 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb08ff-8e5c-41b1-9c88-e4f79b8a33af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:09.776545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb08ff-8e5c-41b1-9c88-e4f79b8a33af admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.776545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb08ff-8e5c-41b1-9c88-e4f79b8a33af admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.776545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb08ff-8e5c-41b1-9c88-e4f79b8a33af admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.776545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb08ff-8e5c-41b1-9c88-e4f79b8a33af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:09.776545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb08ff-8e5c-41b1-9c88-e4f79b8a33af admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:09.786028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1192/2379] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:32:09.789909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:09.799938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad05403-7564-49fe-a06e-cdcf2d81104a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:09.804511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:09.805385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.805675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.805959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] PATH_INFO: `/v3/projects/d8c39cbe757f47119039aba9c4c643ec/users/2fed3b0c9945471b8ae2fe2f54e6de90/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.821440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c39cbe757f47119039aba9c4c643ec, user_id=2fed3b0c9945471b8ae2fe2f54e6de90, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:09.824806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:09.926170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1188/2380] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:32:09 2022] PUT /identity/v3/projects/d8c39cbe757f47119039aba9c4c643ec/users/2fed3b0c9945471b8ae2fe2f54e6de90/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:32:09.935951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfa80e-abc9-4975-8acc-5c537f2d4db2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:09.936248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfa80e-abc9-4975-8acc-5c537f2d4db2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:09.936536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfa80e-abc9-4975-8acc-5c537f2d4db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:09.963123 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d0cfa80e-abc9-4975-8acc-5c537f2d4db2 None None] MFA Rules not processed for user `2fed3b0c9945471b8ae2fe2f54e6de90`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:32:09.965108 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d0cfa80e-abc9-4975-8acc-5c537f2d4db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:10.029161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1193/2381] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:32:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:32:10.033590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:10.039099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:32:10.093994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7137c-1f65-4d6e-ae88-3a2cd1e1ab15 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:10.099391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:10.102441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:10.102441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:10.102441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:10.102441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:10.103195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:10.229441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1189/2382] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:32:10 2022] POST /identity/v3/projects => generated 405 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:32:10.233222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfa80e-abc9-4975-8acc-5c537f2d4db2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:10.244095 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfa80e-abc9-4975-8acc-5c537f2d4db2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:10.302236 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:10.302913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:10.303223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:10.303509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:10.304212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:10.306174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:10.381478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1194/2383] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:32:10 2022] POST /identity/v3/users => generated 309 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:32:10.386280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:10.396689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a88f65-f890-43c6-a030-51a8c9c2804e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:10.401397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:10.402051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:10.402334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:10.402614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:10.403410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:10.408774 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:10.415467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1190/2384] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:32:10.420116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:10.430151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda9b20-57b9-4262-a4ab-694345361b60 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:10.435582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:10.436462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:10.436750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:10.437032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] PATH_INFO: `/v3/projects/17c81f2d34634b3f87da9d30c6d7444d/users/a1259c39fed64623853c112b6a3be46b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:10.451626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c81f2d34634b3f87da9d30c6d7444d, user_id=a1259c39fed64623853c112b6a3be46b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:10.454326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:10.549676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1195/2385] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:32:10 2022] PUT /identity/v3/projects/17c81f2d34634b3f87da9d30c6d7444d/users/a1259c39fed64623853c112b6a3be46b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:32:10.553598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:10.564892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8c90-d101-4817-92e0-2cecfe98630d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:10.627842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e3aca-ac5d-4ca9-b5f7-48d93bcf5d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:10.628398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3aca-ac5d-4ca9-b5f7-48d93bcf5d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:10.628690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3aca-ac5d-4ca9-b5f7-48d93bcf5d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:10.628999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3aca-ac5d-4ca9-b5f7-48d93bcf5d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:10.629733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e3aca-ac5d-4ca9-b5f7-48d93bcf5d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:10.631360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e3aca-ac5d-4ca9-b5f7-48d93bcf5d4b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:10.641397 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1191/2386] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:32:10.645456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:10.655888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930efcf8-53dc-4923-8c70-6600a80bee00 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:10.660468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:10.661337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:10.661623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:10.661901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] PATH_INFO: `/v3/projects/17c81f2d34634b3f87da9d30c6d7444d/users/a1259c39fed64623853c112b6a3be46b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:10.663849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c81f2d34634b3f87da9d30c6d7444d, user_id=a1259c39fed64623853c112b6a3be46b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:10.667377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:10.677740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1196/2387] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:32:10 2022] PUT /identity/v3/projects/17c81f2d34634b3f87da9d30c6d7444d/users/a1259c39fed64623853c112b6a3be46b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:32:10.689658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71966e9-66e5-4c9f-befc-1a97b7fe37ac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:10.689957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71966e9-66e5-4c9f-befc-1a97b7fe37ac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:10.690240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71966e9-66e5-4c9f-befc-1a97b7fe37ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:10.716326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f71966e9-66e5-4c9f-befc-1a97b7fe37ac None None] MFA Rules not processed for user `a1259c39fed64623853c112b6a3be46b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:32:10.717541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f71966e9-66e5-4c9f-befc-1a97b7fe37ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:10.781972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:32:10.785511 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1192/2388] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:32:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:32:20.674088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:INSERT=8 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:32:49.043309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:49.045379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:49.104006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7037c93-c1f2-41e9-8701-53cab0a67f24 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:49.107612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556535d-9e6b-4bff-b010-87726e206d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:49.108256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556535d-9e6b-4bff-b010-87726e206d99 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:49.108544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556535d-9e6b-4bff-b010-87726e206d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:49.108824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556535d-9e6b-4bff-b010-87726e206d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:49.110563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8556535d-9e6b-4bff-b010-87726e206d99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:49.194600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556535d-9e6b-4bff-b010-87726e206d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:49.196009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556535d-9e6b-4bff-b010-87726e206d99 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:49.219076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1197/2389] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:32:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:32:50.031528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71966e9-66e5-4c9f-befc-1a97b7fe37ac None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:50.041593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71966e9-66e5-4c9f-befc-1a97b7fe37ac None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:50.045227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:50.045904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:50.046191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:50.046499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:50.056509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:50.058084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:50.067987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1193/2390] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:32:50 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:32:52.839038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:52.839692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:52.839692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:52.902274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] MFA Rules not processed for user `fef3e61b2a6742ddba146be546996833`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:32:52.903546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:52.959408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:32:52.977704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1198/2391] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 11:32:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:32:53.021750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:53.023264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:53.122271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27866d7-b581-4b06-bbeb-b30fa98f9b42 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:53.127405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fd865-42b2-4a9f-862f-7dccef538e15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f766fee8481a4eccbc56d4af0c2afbf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:53.128066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fd865-42b2-4a9f-862f-7dccef538e15 service swift] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:53.128355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fd865-42b2-4a9f-862f-7dccef538e15 service swift] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:53.128639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fd865-42b2-4a9f-862f-7dccef538e15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:53.137599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fd865-42b2-4a9f-862f-7dccef538e15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:53.139773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fd865-42b2-4a9f-862f-7dccef538e15 service swift] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:53.148016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1194/2392] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 11:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:32:54.109006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:32:54.110457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:54.168117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e08e4e-ed73-4d19-a198-9537e5a7b06f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:32:54.171770 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:32:54.172427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:54.172722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:54.173037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:54.174850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:54.240504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:32:54.241980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:32:54.252536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1199/2393] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:32:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:32:56.045475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d0c09-5b7b-471d-b8ee-d11a3fa5e348 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:32:56.051321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d0c09-5b7b-471d-b8ee-d11a3fa5e348 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:32:56.051321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d0c09-5b7b-471d-b8ee-d11a3fa5e348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:32:56.117351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3c7d0c09-5b7b-471d-b8ee-d11a3fa5e348 None None] MFA Rules not processed for user `fd3eef3ca9da4302a7c2e52acb8155c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:32:56.118853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3c7d0c09-5b7b-471d-b8ee-d11a3fa5e348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:32:56.198475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1195/2394] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:32:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:33:02.026540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:02.028755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:02.041269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027dcc-2e59-4e62-bf8f-3d5794b155df service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:02.046010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:02.046885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:02.047171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:02.047480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] PATH_INFO: `/v3/users/49b35583ea7f489fa62dca85b31a1b64` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:02.048661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b35583ea7f489fa62dca85b31a1b64)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:02.050580 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:02.286222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b35583ea7f489fa62dca85b31a1b64'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.292977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b35583ea7f489fa62dca85b31a1b64'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.293637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b35583ea7f489fa62dca85b31a1b64'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.294239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b35583ea7f489fa62dca85b31a1b64'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.306097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b35583ea7f489fa62dca85b31a1b64'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.392464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1200/2395] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:33:02 2022] DELETE /identity/v3/users/49b35583ea7f489fa62dca85b31a1b64 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:33:02.668346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d0c09-5b7b-471d-b8ee-d11a3fa5e348 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:02.680147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d0c09-5b7b-471d-b8ee-d11a3fa5e348 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:02.720857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:02.754114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:02.754983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:02.755303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:02.755590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] PATH_INFO: `/v3/projects/bf219daa10dd46b58efbfa00e2bf51e2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:02.756771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf219daa10dd46b58efbfa00e2bf51e2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:02.758697 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:02.880061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf219daa10dd46b58efbfa00e2bf51e2', 'request_id': 'req-a58a1382-5ce5-4e74-a685-c4b55be435d3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.885931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf219daa10dd46b58efbfa00e2bf51e2', 'request_id': 'req-a58a1382-5ce5-4e74-a685-c4b55be435d3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.892578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf219daa10dd46b58efbfa00e2bf51e2', 'request_id': 'req-a58a1382-5ce5-4e74-a685-c4b55be435d3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.968271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] The token cache is being invalidate because project bf219daa10dd46b58efbfa00e2bf51e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:33:02.968559 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.969177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:02.970718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1196/2396] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:33:02 2022] DELETE /identity/v3/projects/bf219daa10dd46b58efbfa00e2bf51e2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:33:02.974821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:02.976272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:02.990099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0f9e8-7aa7-4413-927a-0657f69adaf4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:03.000371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:03.052680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:03.053556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:03.054030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:03.054538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] PATH_INFO: `/v3/users/556daa515e71430badb513338c4d7d3e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:03.069981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=556daa515e71430badb513338c4d7d3e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:03.072450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:03.255798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556daa515e71430badb513338c4d7d3e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.262889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556daa515e71430badb513338c4d7d3e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.263665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556daa515e71430badb513338c4d7d3e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.264306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556daa515e71430badb513338c4d7d3e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.277161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e35b9213-48e0-4d1b-b6a5-6a7997c21350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556daa515e71430badb513338c4d7d3e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.375160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1201/2397] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:33:02 2022] DELETE /identity/v3/users/556daa515e71430badb513338c4d7d3e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:33:03.598240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:03.599707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:03.613329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a1382-5ce5-4e74-a685-c4b55be435d3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:03.671411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:03.672275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:03.672561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:03.672841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] PATH_INFO: `/v3/projects/db4f40be37054b608841b8e926f0b553` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:03.683344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db4f40be37054b608841b8e926f0b553)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:03.685288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:03.809206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db4f40be37054b608841b8e926f0b553', 'request_id': 'req-477beccd-74c9-4b39-9ebc-7aec1a6a187c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.810407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:03.815163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db4f40be37054b608841b8e926f0b553', 'request_id': 'req-477beccd-74c9-4b39-9ebc-7aec1a6a187c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.822333 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db4f40be37054b608841b8e926f0b553', 'request_id': 'req-477beccd-74c9-4b39-9ebc-7aec1a6a187c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.928492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] The token cache is being invalidate because project db4f40be37054b608841b8e926f0b553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:33:03.928871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.929500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-477beccd-74c9-4b39-9ebc-7aec1a6a187c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:33:03.931078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1197/2398] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:33:03 2022] DELETE /identity/v3/projects/db4f40be37054b608841b8e926f0b553 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:33:03.936293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f733f0-b6cb-454e-a684-0babbb70b902 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:03.936585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f733f0-b6cb-454e-a684-0babbb70b902 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:03.936871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f733f0-b6cb-454e-a684-0babbb70b902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:03.961936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c7f733f0-b6cb-454e-a684-0babbb70b902 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:33:03.963863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c7f733f0-b6cb-454e-a684-0babbb70b902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:04.038488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1202/2399] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:33:04.046284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec7f5-1d50-44e9-bad3-73d4b900ed62 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.046574 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec7f5-1d50-44e9-bad3-73d4b900ed62 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.046859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec7f5-1d50-44e9-bad3-73d4b900ed62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.066464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-504ec7f5-1d50-44e9-bad3-73d4b900ed62 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:33:04.067790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-504ec7f5-1d50-44e9-bad3-73d4b900ed62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:04.076496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1198/2400] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:33:04.080664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f733f0-b6cb-454e-a684-0babbb70b902 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:04.090381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f733f0-b6cb-454e-a684-0babbb70b902 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:04.095284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:04.095959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.096250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.096533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.097506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:33:04.097658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:04.099354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:04.112281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1203/2401] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:33:04.116716 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ec7f5-1d50-44e9-bad3-73d4b900ed62 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:04.128193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ec7f5-1d50-44e9-bad3-73d4b900ed62 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:04.132883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:04.133614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.134033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.134884 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.135995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:04.138010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:04.208624 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1199/2402] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:33:04 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:33:04.209842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:04.220691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304b2b3-d49e-44d5-9cc4-b8d9be8886a1 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:04.224538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:04.280012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:04.280682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.280963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.281240 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.281930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:04.283874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:04.376034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1204/2403] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:33:04 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:33:04.379873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:04.390560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e9d78-2b1d-41b6-ad3e-b9f07a95d954 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:04.396762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:04.396762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.396762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.396762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.397276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:04.398877 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:04.413441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1200/2404] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:33:04.414615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:04.425442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e2126-9278-4436-b100-630c8e0b3b95 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:04.430528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e1cac-7cbb-4872-b200-213cb8cf54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:04.431968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e1cac-7cbb-4872-b200-213cb8cf54a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.432252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e1cac-7cbb-4872-b200-213cb8cf54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.432661 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e1cac-7cbb-4872-b200-213cb8cf54a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.433467 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e1cac-7cbb-4872-b200-213cb8cf54a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:04.435094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e1cac-7cbb-4872-b200-213cb8cf54a4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:04.445485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1205/2405] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:33:04.449672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:04.460158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd3a16-a6b2-438e-8f04-134ff63a5442 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:04.465039 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:04.466048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.466341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.466625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] PATH_INFO: `/v3/projects/53f26e0accb54e6b85adfa73de12b3af/users/de07a27364da4e948e7f6d5a053fcc0f/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.482510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f26e0accb54e6b85adfa73de12b3af, user_id=de07a27364da4e948e7f6d5a053fcc0f, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:04.485237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:04.580283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1201/2406] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:33:04 2022] PUT /identity/v3/projects/53f26e0accb54e6b85adfa73de12b3af/users/de07a27364da4e948e7f6d5a053fcc0f/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:33:04.585269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6199b4-bb75-4028-88d0-01a593ae924f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.585564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6199b4-bb75-4028-88d0-01a593ae924f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.585845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6199b4-bb75-4028-88d0-01a593ae924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.603905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-5f6199b4-bb75-4028-88d0-01a593ae924f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:33:04.605094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5f6199b4-bb75-4028-88d0-01a593ae924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:04.667704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1206/2407] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:33:04.677118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:04.678855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:04.737287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a3ecf-9df7-4d00-8064-a4e53a0f0f64 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:04.740949 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:04.741596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:04.741885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:04.742171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:04.752771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:04.754198 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:04.763719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1202/2408] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:33:11.056771 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e09f8-2d63-46d3-814e-0793bb9ba664 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:11.057358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e09f8-2d63-46d3-814e-0793bb9ba664 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:11.057358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e09f8-2d63-46d3-814e-0793bb9ba664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:11.099519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f27e09f8-2d63-46d3-814e-0793bb9ba664 None None] MFA Rules not processed for user `de07a27364da4e948e7f6d5a053fcc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:33:11.101057 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f27e09f8-2d63-46d3-814e-0793bb9ba664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:11.192485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1207/2409] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:33:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:33:11.197082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:11.215437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae84e21-e91c-4ef5-819e-1294ed42b09f service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:11.229504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:11.230678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:11.230970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:11.231290 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:11.232104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:11.236454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:11.325524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e09f8-2d63-46d3-814e-0793bb9ba664 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:11.325741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1203/2410] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:33:11 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:33:11.342150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e09f8-2d63-46d3-814e-0793bb9ba664 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:11.355289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:11.426907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:11.427979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:11.427979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:11.427979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:11.428863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:11.431106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:11.514247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1208/2411] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:33:11 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:33:11.517978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:11.530192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae501cd-4931-434f-af54-ff65b9be7286 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:11.538183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:11.538915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:11.539233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:11.539741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:11.540829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:11.542824 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:11.557704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1204/2412] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:33:11.565732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:11.577179 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc94628-96e5-4f0c-8833-ce23d399e7ae admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:11.584332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:11.585397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:11.585599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:11.586031 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] PATH_INFO: `/v3/projects/9f5bf03db2164c529004f2c1885ee86e/users/8fd546d59bae4b439b7d8447b9d7b14d/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:11.605798 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f5bf03db2164c529004f2c1885ee86e, user_id=8fd546d59bae4b439b7d8447b9d7b14d, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:11.610245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:11.713109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1209/2413] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:33:11 2022] PUT /identity/v3/projects/9f5bf03db2164c529004f2c1885ee86e/users/8fd546d59bae4b439b7d8447b9d7b14d/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:33:11.717150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:11.728138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596b61-495a-43d8-a1be-fbc33de8be30 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:11.785804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:11.804630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119034f5-f591-46a4-bf4d-7c253aded8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:11.805311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119034f5-f591-46a4-bf4d-7c253aded8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:11.806509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119034f5-f591-46a4-bf4d-7c253aded8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:11.806683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119034f5-f591-46a4-bf4d-7c253aded8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:11.807577 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119034f5-f591-46a4-bf4d-7c253aded8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:11.809206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119034f5-f591-46a4-bf4d-7c253aded8c7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:11.827867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1205/2414] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:33:11.829175 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:11.842889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f654-6b6f-4bb0-96c6-88d4b52f10b9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:11.847874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:11.848772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:11.849060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:11.849400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] PATH_INFO: `/v3/projects/9f5bf03db2164c529004f2c1885ee86e/users/8fd546d59bae4b439b7d8447b9d7b14d/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:11.851399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f5bf03db2164c529004f2c1885ee86e, user_id=8fd546d59bae4b439b7d8447b9d7b14d, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:11.854872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:11.871497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1210/2415] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:33:11 2022] PUT /identity/v3/projects/9f5bf03db2164c529004f2c1885ee86e/users/8fd546d59bae4b439b7d8447b9d7b14d/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:33:16.916296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:16.916296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:16.916296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:16.953110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] MFA Rules not processed for user `8fd546d59bae4b439b7d8447b9d7b14d`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:33:16.954460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:17.036142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1206/2416] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:33:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:33:17.048325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:17.050932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:17.176935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9a288-5f88-4ccf-a919-462034d0a07a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:17.182958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:17.183662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:17.183948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:17.184227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:17.253314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:17.255021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:17.266718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1211/2417] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:33:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:33:27.029524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=43 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:27.261212 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:33:31.217658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:33:31.219047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:31.331905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8428f48-2a68-4fe9-a69c-cd1db47fc772 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:33:31.336925 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:33:31.338217 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:33:31.338501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:33:31.338776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:33:31.340589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:33:31.351770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:33:31.353653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:33:31.363298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1207/2418] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:33:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:33:41.359138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=50 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:23.630187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:23.631731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:23.697977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf066c-fdcb-4699-abfe-60520f3b7e05 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:23.703207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:23.704066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:23.704362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:23.704639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:23.706411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:23.717804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:23.719597 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:23.730263 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1212/2419] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:34:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:34:27.251982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:27.322129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd33664-100b-450d-9758-02e854cd53b1 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:27.326704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:27.327617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:27.327907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:27.328188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] PATH_INFO: `/v3/users/de07a27364da4e948e7f6d5a053fcc0f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:27.329389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de07a27364da4e948e7f6d5a053fcc0f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:27.331366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:27.589918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de07a27364da4e948e7f6d5a053fcc0f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:27.593016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de07a27364da4e948e7f6d5a053fcc0f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:27.598765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de07a27364da4e948e7f6d5a053fcc0f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:27.599442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de07a27364da4e948e7f6d5a053fcc0f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:27.662349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de07a27364da4e948e7f6d5a053fcc0f'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:27.680491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1208/2420] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:34:27 2022] DELETE /identity/v3/users/de07a27364da4e948e7f6d5a053fcc0f => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:27.955879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:27.967571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8f20e-3d1d-4f00-80cc-14b6e6cb9f77 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:28.033951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:28.035172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:28.035492 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:28.035779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] PATH_INFO: `/v3/projects/53f26e0accb54e6b85adfa73de12b3af` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:28.037037 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f26e0accb54e6b85adfa73de12b3af)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:28.038988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:28.155582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f26e0accb54e6b85adfa73de12b3af', 'request_id': 'req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.161609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f26e0accb54e6b85adfa73de12b3af', 'request_id': 'req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.167947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f26e0accb54e6b85adfa73de12b3af', 'request_id': 'req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.267563 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] The token cache is being invalidate because project 53f26e0accb54e6b85adfa73de12b3af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:34:28.267856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.268470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.270040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1213/2421] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:34:27 2022] DELETE /identity/v3/projects/53f26e0accb54e6b85adfa73de12b3af => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:28.274148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:28.278524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:28.289080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba409b-b2fa-480d-844d-c7a6118ae4a1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:28.345986 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:28.346890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:28.347251 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:28.347539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] PATH_INFO: `/v3/users/8fd546d59bae4b439b7d8447b9d7b14d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:28.348762 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd546d59bae4b439b7d8447b9d7b14d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:28.350757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:28.367021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:28.515987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd546d59bae4b439b7d8447b9d7b14d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.516656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd546d59bae4b439b7d8447b9d7b14d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.523692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd546d59bae4b439b7d8447b9d7b14d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.524341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd546d59bae4b439b7d8447b9d7b14d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.616815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd546d59bae4b439b7d8447b9d7b14d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:28.634747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1209/2422] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:34:28 2022] DELETE /identity/v3/users/8fd546d59bae4b439b7d8447b9d7b14d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:28.892329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:28.893797 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:28.907454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e62b-e8c8-4875-b29e-cec2b321e4b8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:28.935313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:28.966294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:28.967218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:28.967509 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:28.967816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] PATH_INFO: `/v3/projects/9f5bf03db2164c529004f2c1885ee86e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:28.969604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f5bf03db2164c529004f2c1885ee86e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:28.971558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:29.072079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f5bf03db2164c529004f2c1885ee86e', 'request_id': 'req-30358ece-bd90-4f6a-856d-40529d7fec24'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:29.077482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f5bf03db2164c529004f2c1885ee86e', 'request_id': 'req-30358ece-bd90-4f6a-856d-40529d7fec24'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:29.084072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f5bf03db2164c529004f2c1885ee86e', 'request_id': 'req-30358ece-bd90-4f6a-856d-40529d7fec24'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:29.180396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] The token cache is being invalidate because project 9f5bf03db2164c529004f2c1885ee86e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:34:29.180618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:29.181240 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:29.182813 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1214/2423] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:34:28 2022] DELETE /identity/v3/projects/9f5bf03db2164c529004f2c1885ee86e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:38.631145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=9,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:39.177117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:45.345360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:45.346771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:45.413570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f95ae-7611-4716-ae69-1b8641d20226 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:45.460161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8d845-1be9-4f06-8d27-389b55cbecf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:45.460809 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d845-1be9-4f06-8d27-389b55cbecf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:45.461098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d845-1be9-4f06-8d27-389b55cbecf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:45.461377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d845-1be9-4f06-8d27-389b55cbecf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:45.463152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c6a8d845-1be9-4f06-8d27-389b55cbecf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:45.539019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8d845-1be9-4f06-8d27-389b55cbecf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:45.540478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8d845-1be9-4f06-8d27-389b55cbecf5 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:45.551953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1210/2424] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:34:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:34:51.535464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:51.537740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:51.660639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30358ece-bd90-4f6a-856d-40529d7fec24 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:51.665928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:51.666822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:51.667122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:51.667424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] PATH_INFO: `/v3/users/6ae035f9078540adabf1e7578ef06a3d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:51.668853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae035f9078540adabf1e7578ef06a3d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:51.670859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:51.910203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae035f9078540adabf1e7578ef06a3d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:51.917483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae035f9078540adabf1e7578ef06a3d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:51.918721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae035f9078540adabf1e7578ef06a3d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:51.919187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae035f9078540adabf1e7578ef06a3d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:51.933064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae035f9078540adabf1e7578ef06a3d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:52.020735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1215/2425] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:34:51 2022] DELETE /identity/v3/users/6ae035f9078540adabf1e7578ef06a3d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:34:52.025416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d275a1-4d9e-4dbb-8ab1-fdcc6d49639c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:52.025954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d275a1-4d9e-4dbb-8ab1-fdcc6d49639c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:52.026072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d275a1-4d9e-4dbb-8ab1-fdcc6d49639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:52.046079 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-99d275a1-4d9e-4dbb-8ab1-fdcc6d49639c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:52.048238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-99d275a1-4d9e-4dbb-8ab1-fdcc6d49639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:52.091136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:52.116323 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1211/2426] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:34:52.126500 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:52.127942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:52.198895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31213a6-e10e-49bd-94eb-3528b2bb13c0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:52.203933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:52.203933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:52.204189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:52.204456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:52.213778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:52.215613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:52.227150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1216/2427] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:34:52.433078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:52.433078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:52.433078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:52.450781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] MFA Rules not processed for user `fd3eef3ca9da4302a7c2e52acb8155c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:52.452278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:52.520480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1212/2428] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:34:52.695299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:52.699048 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:52.710931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934eb3-45cd-40b7-b1a7-ada376f8265e service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:52.715905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:52.716766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:52.717050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:52.717366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] PATH_INFO: `/v3/projects/bb07d6b829814af5af6afa53a1e4a3ad` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:52.718579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb07d6b829814af5af6afa53a1e4a3ad)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:52.720867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:52.745060 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:52.861838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:52.872435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:52.876116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb07d6b829814af5af6afa53a1e4a3ad', 'request_id': 'req-21213cd2-91af-40c1-9bef-69229d31baa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:52.876605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6807a2-6f78-451e-8cfa-58a12853d9c2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:52.881626 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb07d6b829814af5af6afa53a1e4a3ad', 'request_id': 'req-21213cd2-91af-40c1-9bef-69229d31baa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:52.888532 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb07d6b829814af5af6afa53a1e4a3ad', 'request_id': 'req-21213cd2-91af-40c1-9bef-69229d31baa3'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:52.936881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec385b-b4f4-49b9-983c-2cde20a03a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:52.937580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec385b-b4f4-49b9-983c-2cde20a03a9c service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:52.938061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec385b-b4f4-49b9-983c-2cde20a03a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:52.938420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec385b-b4f4-49b9-983c-2cde20a03a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:52.940406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cfec385b-b4f4-49b9-983c-2cde20a03a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:52.946777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] The token cache is being invalidate because project bb07d6b829814af5af6afa53a1e4a3ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:34:52.954656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:52.954656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-21213cd2-91af-40c1-9bef-69229d31baa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:52.954656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1217/2429] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:34:52 2022] DELETE /identity/v3/projects/bb07d6b829814af5af6afa53a1e4a3ad => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:52.955012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec385b-b4f4-49b9-983c-2cde20a03a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:52.955012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec385b-b4f4-49b9-983c-2cde20a03a9c service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:52.955607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339574e9-f78f-4d77-842b-015e95c96fe9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:52.955955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339574e9-f78f-4d77-842b-015e95c96fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:52.956238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339574e9-f78f-4d77-842b-015e95c96fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:52.961190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1213/2430] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:34:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:34:52.977002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-339574e9-f78f-4d77-842b-015e95c96fe9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:52.978235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-339574e9-f78f-4d77-842b-015e95c96fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:53.040975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1218/2431] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:34:53.048804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a364a2-d713-425d-a69d-8e11e0075fc4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.049434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a364a2-d713-425d-a69d-8e11e0075fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.049434 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a364a2-d713-425d-a69d-8e11e0075fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.072938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b6a364a2-d713-425d-a69d-8e11e0075fc4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:53.074229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b6a364a2-d713-425d-a69d-8e11e0075fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:53.083487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1214/2432] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:34:53.087876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339574e9-f78f-4d77-842b-015e95c96fe9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:53.100690 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339574e9-f78f-4d77-842b-015e95c96fe9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:53.105538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:53.106219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.106511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.106825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.107643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:34:53.107959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:53.109635 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:53.122791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1219/2433] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:34:53.127547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a364a2-d713-425d-a69d-8e11e0075fc4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:53.137436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a364a2-d713-425d-a69d-8e11e0075fc4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:53.145078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:53.145771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.146062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.146346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.147059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:53.149066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:53.232918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1215/2434] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:34:53 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:34:53.237754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:53.248691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ade8b-4632-4dc8-8af2-2af1182935af admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:53.308083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:53.308759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.309047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.309333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.310033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:53.311999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:53.406994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1220/2435] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:34:53 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:34:53.410016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:53.420450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08539b-277c-462a-8c05-3590fdaf0d2b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:53.425158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:53.428618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.428618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.428618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.428618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:53.429035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:53.441380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1216/2436] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:34:53.444858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:53.455843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067e35d-6dd2-4ff1-9105-c298ade0cb7c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:53.460474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57920c3c-e8e6-42b9-91e7-2c6f218e7ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:53.461129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920c3c-e8e6-42b9-91e7-2c6f218e7ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.461419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920c3c-e8e6-42b9-91e7-2c6f218e7ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.461700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920c3c-e8e6-42b9-91e7-2c6f218e7ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.462426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57920c3c-e8e6-42b9-91e7-2c6f218e7ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:53.464072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57920c3c-e8e6-42b9-91e7-2c6f218e7ccc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:53.474032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1221/2437] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:34:53.478058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:53.488970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbc4c3-1687-4468-8954-346d86a40000 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:53.493903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:53.494788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.495102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.495413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] PATH_INFO: `/v3/projects/a692be969a3c4387933f0bc5349236c7/users/c8f99704bcc0451bb08989609425dbcc/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.510303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a692be969a3c4387933f0bc5349236c7, user_id=c8f99704bcc0451bb08989609425dbcc, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:53.513075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:53.610832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1217/2438] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:34:53 2022] PUT /identity/v3/projects/a692be969a3c4387933f0bc5349236c7/users/c8f99704bcc0451bb08989609425dbcc/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:53.620982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:53.621276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:53.621558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:53.644382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:53.650934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] MFA Rules not processed for user `c8f99704bcc0451bb08989609425dbcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:53.652246 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:53.717949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1222/2439] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:34:54.365978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:54.367764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:54.401450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:54.445140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd286c-e07a-4e47-9601-a93b0a96a51f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:54.450472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:54.451635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:54.451985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:54.452272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:54.454676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:54.466613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:54.468255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:54.480773 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1218/2440] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:34:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:34:54.930040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:54.931541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:54.988523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:54.990225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:55.018754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0963a5e-ab04-47e3-bfbd-8672b881f610 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:55.023791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:55.024664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:55.024968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:55.025211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] PATH_INFO: `/v3/users/2fed3b0c9945471b8ae2fe2f54e6de90` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:55.026396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fed3b0c9945471b8ae2fe2f54e6de90)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:55.028372 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:55.049593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016265b5-d8b1-4644-b380-f270a9cebb87 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:55.053343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa5d61-17e4-4988-95ca-d8f0aae8328b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:55.053992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa5d61-17e4-4988-95ca-d8f0aae8328b service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:55.054283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa5d61-17e4-4988-95ca-d8f0aae8328b service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:55.054567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa5d61-17e4-4988-95ca-d8f0aae8328b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:55.063519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa5d61-17e4-4988-95ca-d8f0aae8328b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:55.066204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa5d61-17e4-4988-95ca-d8f0aae8328b service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:55.073848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1219/2441] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:34:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:34:55.213896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fed3b0c9945471b8ae2fe2f54e6de90'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:55.221131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fed3b0c9945471b8ae2fe2f54e6de90'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:55.221823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fed3b0c9945471b8ae2fe2f54e6de90'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:55.222438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fed3b0c9945471b8ae2fe2f54e6de90'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:55.234088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fed3b0c9945471b8ae2fe2f54e6de90'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:55.334473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1223/2442] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:34:54 2022] DELETE /identity/v3/users/2fed3b0c9945471b8ae2fe2f54e6de90 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:55.339010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:55.339337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:55.339629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:55.357566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:55.358817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:55.422897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1220/2443] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:34:55.432683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:55.434098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:55.456832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:55.494440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ddd82-a3b3-4845-9466-d2c7adbf6a6b admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:55.503125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:55.503125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:55.503125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:55.503125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:55.513872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:55.518442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:55.528345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1224/2444] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:34:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:34:56.017998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:56.019656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:56.033898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe0fde-6a59-4e0e-8741-e0c7d514a632 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:56.039061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:56.043248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:56.043248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:56.043248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] PATH_INFO: `/v3/projects/d8c39cbe757f47119039aba9c4c643ec` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:56.043248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c39cbe757f47119039aba9c4c643ec)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:56.043248 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:56.160052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c39cbe757f47119039aba9c4c643ec', 'request_id': 'req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.164023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:56.166212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c39cbe757f47119039aba9c4c643ec', 'request_id': 'req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.173297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c39cbe757f47119039aba9c4c643ec', 'request_id': 'req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.273031 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] The token cache is being invalidate because project d8c39cbe757f47119039aba9c4c643ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:34:56.273349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.273970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.275554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1221/2445] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:34:56 2022] DELETE /identity/v3/projects/d8c39cbe757f47119039aba9c4c643ec => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:56.279730 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:56.290316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed9c07-f682-424e-8e91-df985cb72d76 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:56.349402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:56.350272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:56.350560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:56.350840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] PATH_INFO: `/v3/users/a1259c39fed64623853c112b6a3be46b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:56.352064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1259c39fed64623853c112b6a3be46b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:56.354001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:56.549676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1259c39fed64623853c112b6a3be46b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.556619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1259c39fed64623853c112b6a3be46b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.557287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1259c39fed64623853c112b6a3be46b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.557955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1259c39fed64623853c112b6a3be46b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.573365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7616b0f8-3b53-489c-9e03-4637699fe42c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1259c39fed64623853c112b6a3be46b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:56.663155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1225/2446] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:34:56 2022] DELETE /identity/v3/users/a1259c39fed64623853c112b6a3be46b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:57.352930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:57.354747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:57.369157 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05661c-8f1b-4cdd-9e7b-863fe8e57985 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:57.431588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:57.432479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:57.433540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:57.433540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] PATH_INFO: `/v3/projects/17c81f2d34634b3f87da9d30c6d7444d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:57.434492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c81f2d34634b3f87da9d30c6d7444d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:57.436428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:57.541794 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c81f2d34634b3f87da9d30c6d7444d', 'request_id': 'req-e5eade79-a079-4486-b807-9c890a853ae4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:57.547653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c81f2d34634b3f87da9d30c6d7444d', 'request_id': 'req-e5eade79-a079-4486-b807-9c890a853ae4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:57.554263 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c81f2d34634b3f87da9d30c6d7444d', 'request_id': 'req-e5eade79-a079-4486-b807-9c890a853ae4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:57.616890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] The token cache is being invalidate because project 17c81f2d34634b3f87da9d30c6d7444d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:34:57.617227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:57.617836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e5eade79-a079-4486-b807-9c890a853ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:34:57.619421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1222/2447] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:34:57 2022] DELETE /identity/v3/projects/17c81f2d34634b3f87da9d30c6d7444d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:57.624412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759680-7d98-46e7-9bc4-9f59c1562c67 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:57.624937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759680-7d98-46e7-9bc4-9f59c1562c67 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:57.625588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f759680-7d98-46e7-9bc4-9f59c1562c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:57.643565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0f759680-7d98-46e7-9bc4-9f59c1562c67 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:57.644821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0f759680-7d98-46e7-9bc4-9f59c1562c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:57.689344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:57.715133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1226/2448] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:34:57.723068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7ac3-4af1-42e6-bcdb-2a86c82b5aa8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:57.723526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7ac3-4af1-42e6-bcdb-2a86c82b5aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:57.723911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7ac3-4af1-42e6-bcdb-2a86c82b5aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:57.744842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-a16e7ac3-4af1-42e6-bcdb-2a86c82b5aa8 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:57.746151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a16e7ac3-4af1-42e6-bcdb-2a86c82b5aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:57.755982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1223/2449] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:34:57.760105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759680-7d98-46e7-9bc4-9f59c1562c67 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:57.770030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f759680-7d98-46e7-9bc4-9f59c1562c67 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:57.774528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:57.775224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:57.775513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:57.775788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:57.776534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:34:57.776867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:57.778553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:57.791380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1227/2450] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:34:57.795928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e7ac3-4af1-42e6-bcdb-2a86c82b5aa8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:57.805798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e7ac3-4af1-42e6-bcdb-2a86c82b5aa8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:57.810460 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:57.811112 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:57.811430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:57.811713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:57.812578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:57.814389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:57.935663 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1224/2451] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:34:57 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:34:57.941997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:57.952802 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7b2ec-ee40-41e9-9d47-75929fddb1ea admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.010028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.010722 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.011011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.011457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.012194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.014129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.089013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1228/2452] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:34:57 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:34:58.098974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:58.110067 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2aed1-6334-4a17-bcd3-496b9cef86a8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.114936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.115634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.115930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.116213 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.116945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.118544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.130272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1225/2453] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:34:58.134362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:58.145162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611d8c5-1d6d-4ba3-8c18-065d2fbc5828 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.149753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf49c93-ae9e-4d38-85fe-ac30c9245c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.150403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf49c93-ae9e-4d38-85fe-ac30c9245c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.150692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf49c93-ae9e-4d38-85fe-ac30c9245c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.150968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf49c93-ae9e-4d38-85fe-ac30c9245c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.151732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf49c93-ae9e-4d38-85fe-ac30c9245c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.153328 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf49c93-ae9e-4d38-85fe-ac30c9245c23 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.163814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1229/2454] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:34:58.167739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:58.178351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd31a-4897-4706-af17-4e4e731de977 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.182956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.183866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.184153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.184433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] PATH_INFO: `/v3/projects/07e55b3cb5cd49eab46e2f7b5d6c267b/users/cfbf9999c62d4d1b98f334921779777d/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.198636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07e55b3cb5cd49eab46e2f7b5d6c267b, user_id=cfbf9999c62d4d1b98f334921779777d, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.201326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.307813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1226/2455] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:34:58 2022] PUT /identity/v3/projects/07e55b3cb5cd49eab46e2f7b5d6c267b/users/cfbf9999c62d4d1b98f334921779777d/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:58.317909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea043b39-898f-4f95-ad73-c268a7238097 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.318201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea043b39-898f-4f95-ad73-c268a7238097 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.318481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea043b39-898f-4f95-ad73-c268a7238097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.346842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ea043b39-898f-4f95-ad73-c268a7238097 None None] MFA Rules not processed for user `cfbf9999c62d4d1b98f334921779777d`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:58.347991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ea043b39-898f-4f95-ad73-c268a7238097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:58.414099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1230/2456] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:34:58.418705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:58.430327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:58.482907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc2ef1-d5a1-4708-9f6f-0c9abf2a3c7f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.488113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.489035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.489518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.490024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.491145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.493218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.610952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1227/2457] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:34:58 2022] POST /identity/v3/projects => generated 367 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:34:58.612059 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea043b39-898f-4f95-ad73-c268a7238097 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:58.624854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea043b39-898f-4f95-ad73-c268a7238097 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.661514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:34:58.685270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.685952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.686272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.686553 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.687357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.693431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.803272 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1231/2458] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:34:58 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:34:58.804487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:58.815454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d78c0b-7eee-4720-bde1-727534381082 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.820703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.821473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.821823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.822167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.822970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.824896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.836644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1228/2459] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:34:58.840658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:58.854188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cbd1a-75a9-4445-a72c-9245d9f59850 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:58.858517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:58.864137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:58.864137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:58.864137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] PATH_INFO: `/v3/projects/4003d34eae4244cf9ea1516585c095bc/users/4328b59adc7b4dd5a3a970563c4fe53b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:58.877380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4003d34eae4244cf9ea1516585c095bc, user_id=4328b59adc7b4dd5a3a970563c4fe53b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:58.888765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:58.986311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1232/2460] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:34:58 2022] PUT /identity/v3/projects/4003d34eae4244cf9ea1516585c095bc/users/4328b59adc7b4dd5a3a970563c4fe53b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:58.991385 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:59.001790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f53ebf-7eeb-4c13-84e7-00dd59552ce1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:59.064381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4495333-8799-44e0-adc5-3581026f5ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:59.065151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4495333-8799-44e0-adc5-3581026f5ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:59.065442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4495333-8799-44e0-adc5-3581026f5ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:59.065771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4495333-8799-44e0-adc5-3581026f5ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:59.066505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4495333-8799-44e0-adc5-3581026f5ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:59.068179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4495333-8799-44e0-adc5-3581026f5ba6 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:59.079604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1229/2461] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:34:59.084003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:34:59.095196 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026e518-2d7d-464c-b661-2b5af6d1d398 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:34:59.099817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:34:59.100700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:59.100985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:59.101269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] PATH_INFO: `/v3/projects/4003d34eae4244cf9ea1516585c095bc/users/4328b59adc7b4dd5a3a970563c4fe53b/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:59.103275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4003d34eae4244cf9ea1516585c095bc, user_id=4328b59adc7b4dd5a3a970563c4fe53b, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:34:59.105941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:34:59.121517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1233/2462] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:34:59 2022] PUT /identity/v3/projects/4003d34eae4244cf9ea1516585c095bc/users/4328b59adc7b4dd5a3a970563c4fe53b/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:34:59.131296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5717f-cbe1-4957-86de-bead7bcba3c8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:34:59.131673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5717f-cbe1-4957-86de-bead7bcba3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:34:59.131957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5717f-cbe1-4957-86de-bead7bcba3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:34:59.159995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d5a5717f-cbe1-4957-86de-bead7bcba3c8 None None] MFA Rules not processed for user `4328b59adc7b4dd5a3a970563c4fe53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:34:59.161225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d5a5717f-cbe1-4957-86de-bead7bcba3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:34:59.237019 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1230/2463] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:34:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:35:07.000435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:35:07.002332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:07.080943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56ba01-f898-4270-9b22-68fd988ddf21 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:35:07.087657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:35:07.088316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:07.088605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:07.088884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:07.090791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:07.103778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:35:07.105366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:35:07.116126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1234/2464] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:35:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:35:09.233538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:17.111510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:35:42.759893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5717f-cbe1-4957-86de-bead7bcba3c8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:35:42.785250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5717f-cbe1-4957-86de-bead7bcba3c8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:35:42.791707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:35:42.791707 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:35:42.791994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:35:42.792126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:35:42.793818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:35:42.854194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:35:42.855825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:35:42.865450 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1231/2465] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:35:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:35:52.861429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=32 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:30.321219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:30.322700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:30.378885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd75806-1b88-4b7d-9ae8-b201c3155cd9 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:30.382484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:30.383126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:30.383457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:30.383738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:30.385485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:30.394887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:30.396367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:30.404264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1235/2466] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:40.400698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=26 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:51.715072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:51.716505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:51.787694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b78e-b737-4591-925d-a3da149218ad service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:51.791393 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:51.792066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:51.792356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:51.792639 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:51.794419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:51.854633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:51.856163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:51.865656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1232/2467] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:51.929689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:51.940218 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e94e3d-20ac-4f17-98a7-3a1f4f9846ac service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:51.943869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:51.944513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:51.944803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:51.945080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:51.955126 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:51.956804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:51.966223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1236/2468] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:52.854652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:52.864051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3481761-1f5e-43f2-b265-98b8644e5d69 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:52.867932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f80d5-faeb-436e-991b-c2b8f741e309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:52.868580 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f80d5-faeb-436e-991b-c2b8f741e309 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:52.868865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f80d5-faeb-436e-991b-c2b8f741e309 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:52.869143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f80d5-faeb-436e-991b-c2b8f741e309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:52.945267 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f80d5-faeb-436e-991b-c2b8f741e309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:52.946713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f80d5-faeb-436e-991b-c2b8f741e309 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:52.957458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1233/2469] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:56.934545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:57.006441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de587a-7c82-4a7c-825c-39f4ee7422d5 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:57.011047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:57.011941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:57.012230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:57.012512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] PATH_INFO: `/v3/users/cfbf9999c62d4d1b98f334921779777d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:57.013712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbf9999c62d4d1b98f334921779777d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:57.015739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:57.186749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbf9999c62d4d1b98f334921779777d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:57.193544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbf9999c62d4d1b98f334921779777d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:57.195098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbf9999c62d4d1b98f334921779777d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:57.195098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbf9999c62d4d1b98f334921779777d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:57.206562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbf9999c62d4d1b98f334921779777d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:57.276153 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1237/2470] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:36:56 2022] DELETE /identity/v3/users/cfbf9999c62d4d1b98f334921779777d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:36:57.280784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:57.286893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:57.286893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:57.300154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:57.301360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:57.344920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:57.363370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1234/2471] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:57.372938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:57.374319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:57.430487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccfaab-6f63-4bd4-8614-b34a72de09a4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:57.434121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:57.434734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:57.435018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:57.435332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:57.444568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:57.445933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:57.455962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1238/2472] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:36:57.866343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:57.868625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:57.883301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b2a8-1e9a-4799-a048-109189435ffd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:57.888073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:57.889795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:57.891957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:57.891957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] PATH_INFO: `/v3/projects/07e55b3cb5cd49eab46e2f7b5d6c267b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:57.893582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07e55b3cb5cd49eab46e2f7b5d6c267b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:57.895899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:57.994896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07e55b3cb5cd49eab46e2f7b5d6c267b', 'request_id': 'req-c7f84b26-9cfc-440f-89e2-e18756e0e764'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.001659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07e55b3cb5cd49eab46e2f7b5d6c267b', 'request_id': 'req-c7f84b26-9cfc-440f-89e2-e18756e0e764'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.008437 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07e55b3cb5cd49eab46e2f7b5d6c267b', 'request_id': 'req-c7f84b26-9cfc-440f-89e2-e18756e0e764'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.092827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] The token cache is being invalidate because project 07e55b3cb5cd49eab46e2f7b5d6c267b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:36:58.093123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.093813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.095521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1235/2473] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:36:57 2022] DELETE /identity/v3/projects/07e55b3cb5cd49eab46e2f7b5d6c267b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:36:58.099471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:58.109987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b227b8-dc6c-467d-b102-fd1c0693aedc service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:58.115129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:58.165738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:58.166653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:58.167129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:58.167476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] PATH_INFO: `/v3/users/4328b59adc7b4dd5a3a970563c4fe53b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:58.169163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4328b59adc7b4dd5a3a970563c4fe53b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:58.171336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:58.390431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4328b59adc7b4dd5a3a970563c4fe53b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.397371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4328b59adc7b4dd5a3a970563c4fe53b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.398100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4328b59adc7b4dd5a3a970563c4fe53b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.398719 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4328b59adc7b4dd5a3a970563c4fe53b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.411472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-8bb3663a-1ce7-4db1-8a33-48d5c8a59a52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4328b59adc7b4dd5a3a970563c4fe53b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:58.502645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1239/2474] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:36:58 2022] DELETE /identity/v3/users/4328b59adc7b4dd5a3a970563c4fe53b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:36:58.967096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:58.970139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:58.985141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f84b26-9cfc-440f-89e2-e18756e0e764 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:59.053659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:59.054651 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.055006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.055520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] PATH_INFO: `/v3/projects/4003d34eae4244cf9ea1516585c095bc` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.056746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4003d34eae4244cf9ea1516585c095bc)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:59.058660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:59.178717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4003d34eae4244cf9ea1516585c095bc', 'request_id': 'req-b9336240-cd87-44c4-b35f-17c2d1cc70e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:59.184074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4003d34eae4244cf9ea1516585c095bc', 'request_id': 'req-b9336240-cd87-44c4-b35f-17c2d1cc70e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:59.190684 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4003d34eae4244cf9ea1516585c095bc', 'request_id': 'req-b9336240-cd87-44c4-b35f-17c2d1cc70e3'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:59.271052 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:59.275746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] The token cache is being invalidate because project 4003d34eae4244cf9ea1516585c095bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:36:59.276114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:59.276718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-b9336240-cd87-44c4-b35f-17c2d1cc70e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:36:59.278298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1236/2475] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:36:58 2022] DELETE /identity/v3/projects/4003d34eae4244cf9ea1516585c095bc => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:36:59.283289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f0af-4624-4cb9-90ea-0c58444d0166 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.283780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f0af-4624-4cb9-90ea-0c58444d0166 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.283888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f0af-4624-4cb9-90ea-0c58444d0166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.302150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c347f0af-4624-4cb9-90ea-0c58444d0166 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:59.303408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c347f0af-4624-4cb9-90ea-0c58444d0166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:59.362801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1240/2476] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:59.370653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6147a-04a2-4997-bccf-6e4a8408727e None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.370968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6147a-04a2-4997-bccf-6e4a8408727e None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.371362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6147a-04a2-4997-bccf-6e4a8408727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.392939 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-32f6147a-04a2-4997-bccf-6e4a8408727e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:59.394708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-32f6147a-04a2-4997-bccf-6e4a8408727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:59.402911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1237/2477] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:59.407140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347f0af-4624-4cb9-90ea-0c58444d0166 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:59.417042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347f0af-4624-4cb9-90ea-0c58444d0166 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:59.421606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:59.422652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.422944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.423261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.424002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:36:59.424315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:59.425913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:59.437868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1241/2478] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:36:59.441892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6147a-04a2-4997-bccf-6e4a8408727e None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:59.451463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6147a-04a2-4997-bccf-6e4a8408727e None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:59.456710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:59.457356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.457643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.457922 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.458608 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:59.460535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:59.540183 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1238/2479] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:36:59 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:36:59.544513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:59.555425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bdb5-cfa4-4877-a7df-b5af3d4364a7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:59.567564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:36:59.612024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:59.612699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.612988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.613268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.613985 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:59.615970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:59.676930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1242/2480] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:36:59 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:36:59.680043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:59.691526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21d41c-9115-44ba-8991-e0e36d447ae1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:59.696079 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:59.696735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.697022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.697336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.698061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:59.699725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:59.710240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1239/2481] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:36:59.714094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:59.724310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9615e5-a326-48e2-9a03-9a0b88074c6f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:59.728924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f44f9-56d5-447c-abde-f9385f8fb9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:59.729574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f44f9-56d5-447c-abde-f9385f8fb9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.729860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f44f9-56d5-447c-abde-f9385f8fb9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.730138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f44f9-56d5-447c-abde-f9385f8fb9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.730861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f44f9-56d5-447c-abde-f9385f8fb9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:59.732490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f44f9-56d5-447c-abde-f9385f8fb9e7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:59.743361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1243/2482] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:36:59.747300 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:36:59.757822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6d08-2edd-42f5-995f-ef5d7fda8af3 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:36:59.762458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:36:59.763381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.763701 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.763983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] PATH_INFO: `/v3/projects/bffdd5a183af4c379f7f86f46f39b70b/users/e40675ea75c04ad297d47623d2f60511/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.778334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bffdd5a183af4c379f7f86f46f39b70b, user_id=e40675ea75c04ad297d47623d2f60511, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:36:59.781047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:36:59.850915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1240/2483] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:36:59 2022] PUT /identity/v3/projects/bffdd5a183af4c379f7f86f46f39b70b/users/e40675ea75c04ad297d47623d2f60511/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:36:59.862020 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678df8-3165-4567-a6a9-79fc084f2698 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:36:59.862321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678df8-3165-4567-a6a9-79fc084f2698 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:36:59.862607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678df8-3165-4567-a6a9-79fc084f2698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:36:59.889524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-91678df8-3165-4567-a6a9-79fc084f2698 None None] MFA Rules not processed for user `e40675ea75c04ad297d47623d2f60511`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:36:59.890921 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-91678df8-3165-4567-a6a9-79fc084f2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:36:59.954998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1244/2484] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:36:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:36:59.959476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.024363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200064e-b521-44b2-8952-264db2e9da81 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.031734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.031734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.031734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.031734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.032185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.034155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.121830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1241/2485] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:37:00.125065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91678df8-3165-4567-a6a9-79fc084f2698 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.136201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91678df8-3165-4567-a6a9-79fc084f2698 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.201011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.201710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.202002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.202287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.203145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.205958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.316786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1245/2486] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:37:00 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:37:00.320750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.331905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc906f6-91df-49d1-ac85-9c285c430684 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.337126 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.337927 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.338258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.338582 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.339413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.341221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.352484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1242/2487] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:37:00.356260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.367051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a3549-e4d9-4afa-aaab-4f4da20c8156 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.371747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d800d3-66c6-40f9-a5c4-53a9cd0e3c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.372401 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d800d3-66c6-40f9-a5c4-53a9cd0e3c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.372692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d800d3-66c6-40f9-a5c4-53a9cd0e3c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.372977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d800d3-66c6-40f9-a5c4-53a9cd0e3c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.373705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d800d3-66c6-40f9-a5c4-53a9cd0e3c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.375754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d800d3-66c6-40f9-a5c4-53a9cd0e3c11 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.385934 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1246/2488] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:37:00.390002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.400616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e020ee-dcad-43f0-8732-c277e3dff802 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.406042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.407566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.407857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.408142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] PATH_INFO: `/v3/projects/260b96e94c394d8b871cc592da906fbf/users/336995499cb642c7b8869b120f83b68e/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.423705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] RBAC: Authorizing `identity:create_grant(project_id=260b96e94c394d8b871cc592da906fbf, user_id=336995499cb642c7b8869b120f83b68e, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.426490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.504655 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1243/2489] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:37:00 2022] PUT /identity/v3/projects/260b96e94c394d8b871cc592da906fbf/users/336995499cb642c7b8869b120f83b68e/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:37:00.564961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195707a-85ae-4ef8-8ad1-4ac23fb93f8f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.565283 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195707a-85ae-4ef8-8ad1-4ac23fb93f8f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.565596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195707a-85ae-4ef8-8ad1-4ac23fb93f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.571631 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:00.595083 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3195707a-85ae-4ef8-8ad1-4ac23fb93f8f None None] MFA Rules not processed for user `336995499cb642c7b8869b120f83b68e`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:00.596368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3195707a-85ae-4ef8-8ad1-4ac23fb93f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:00.663785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1247/2490] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:37:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:00.668392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.716140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:00.731959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73fe96-1eb7-4d31-b0e1-f9d60f51200d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.737074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.737727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.738014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.738294 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.738981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.740937 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.831712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1244/2491] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:37:00 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:37:00.835927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195707a-85ae-4ef8-8ad1-4ac23fb93f8f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:00.847818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195707a-85ae-4ef8-8ad1-4ac23fb93f8f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:00.908324 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:00.909040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:00.909327 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:00.909610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:00.910319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:00.912302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:00.999590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1248/2492] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:37:00 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:37:01.003752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:01.013999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec7aaf-4f1a-4258-b36c-c8945f93d7d8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:01.020959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:01.021631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:01.021919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:01.022196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:01.022930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:01.024599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:01.036196 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1245/2493] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:37:01.040530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:01.051091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec932c-2b56-4058-a484-cd0c7b6abe40 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:01.055759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:01.056640 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:01.056928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:01.057262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] PATH_INFO: `/v3/projects/bcce9424860140feaa2ca69a4a04e63d/users/aeca0908494c408b821ba3ecada46cbb/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:01.072480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcce9424860140feaa2ca69a4a04e63d, user_id=aeca0908494c408b821ba3ecada46cbb, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:01.075161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:01.148530 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1249/2494] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:37:01 2022] PUT /identity/v3/projects/bcce9424860140feaa2ca69a4a04e63d/users/aeca0908494c408b821ba3ecada46cbb/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:37:01.152413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:01.163062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d1624-9186-414e-b5cf-f53c3ef9e32f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:01.218257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ca1d8-3d46-41c3-9970-0dc9b045821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:01.219710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ca1d8-3d46-41c3-9970-0dc9b045821f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:01.219996 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ca1d8-3d46-41c3-9970-0dc9b045821f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:01.220306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ca1d8-3d46-41c3-9970-0dc9b045821f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:01.221040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ca1d8-3d46-41c3-9970-0dc9b045821f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:01.222611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ca1d8-3d46-41c3-9970-0dc9b045821f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:01.232503 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1246/2495] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:37:01.237143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:01.247603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d1693-49e8-4a08-9f89-fe6a63cf020e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:01.252464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:01.253336 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:01.253619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:01.253896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] PATH_INFO: `/v3/projects/bcce9424860140feaa2ca69a4a04e63d/users/aeca0908494c408b821ba3ecada46cbb/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:01.255886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcce9424860140feaa2ca69a4a04e63d, user_id=aeca0908494c408b821ba3ecada46cbb, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:01.258528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:01.269064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1250/2496] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:37:01 2022] PUT /identity/v3/projects/bcce9424860140feaa2ca69a4a04e63d/users/aeca0908494c408b821ba3ecada46cbb/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:37:01.279078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba8b84-7427-4994-9ad1-408766ea7083 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:01.279396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba8b84-7427-4994-9ad1-408766ea7083 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:01.279705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba8b84-7427-4994-9ad1-408766ea7083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:01.307363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-0bba8b84-7427-4994-9ad1-408766ea7083 None None] MFA Rules not processed for user `aeca0908494c408b821ba3ecada46cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:37:01.308576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0bba8b84-7427-4994-9ad1-408766ea7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:01.378733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1247/2497] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:37:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:37:01.389412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:01.391903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:01.437962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:01.448586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6306e-d47a-4a13-9bf0-aca07811bf20 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:01.452485 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:01.453733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:01.454018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:01.454297 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:01.463370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:01.464811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:01.474886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1251/2498] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:02.665981 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba8b84-7427-4994-9ad1-408766ea7083 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:02.675672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba8b84-7427-4994-9ad1-408766ea7083 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:02.679261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:02.679904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:02.680194 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:02.680476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:02.719840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:02.737992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:02.739471 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:02.748376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1248/2499] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:11.492773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:12.787733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=10 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:37:48.822684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:37:48.831503 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9e2e0-45b5-4a80-b5ac-286de6d911be service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:37:48.835133 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:37:48.836948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:37:48.836948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:37:48.836948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:37:48.838484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:37:48.898803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:37:48.900271 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:37:48.909754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1252/2500] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:37:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:37:58.908131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=28 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:33.572692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:33.574140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:33.644976 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa6b2c-39c4-4ff5-adb0-d3f035f39e34 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:33.649865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:33.650561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:33.650844 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:33.651122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:33.652899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:33.664704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:33.666272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:33.676711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1249/2501] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:43.671841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=32 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:54.636332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:54.637723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:54.705398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ec04b-8145-4a82-9823-f69807e73859 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:54.709452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:54.710090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:54.710371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:54.710646 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:54.712450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:54.795752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:54.797190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:54.816378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1253/2502] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:38:58.295818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:58.297966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:58.312648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a30521-2f2d-4080-942a-fa1e25c1e7c2 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:58.317428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:58.318338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:58.318630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:58.318915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] PATH_INFO: `/v3/users/5d079dc5e2b142c7a088648f79d1bb0b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:58.320257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d079dc5e2b142c7a088648f79d1bb0b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:58.322321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:58.541170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d079dc5e2b142c7a088648f79d1bb0b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:58.541780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d079dc5e2b142c7a088648f79d1bb0b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:58.548746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d079dc5e2b142c7a088648f79d1bb0b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:58.549409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d079dc5e2b142c7a088648f79d1bb0b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:58.612534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dbfa96d9-4e3c-4e3a-845a-9e4b8581af40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d079dc5e2b142c7a088648f79d1bb0b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:58.631920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1250/2503] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:38:58 2022] DELETE /identity/v3/users/5d079dc5e2b142c7a088648f79d1bb0b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:38:58.888144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:58.899529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd75f6-5137-44d1-aaa6-45170a8cfb5e service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:58.967198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:58.968090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:58.968371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:58.968647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] PATH_INFO: `/v3/projects/c738995fef2342e99240de801574307d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:58.969819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c738995fef2342e99240de801574307d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:58.971800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:58.983233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:59.086961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c738995fef2342e99240de801574307d', 'request_id': 'req-9974c8cf-a299-4f24-993c-23838f18d4ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:59.092521 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c738995fef2342e99240de801574307d', 'request_id': 'req-9974c8cf-a299-4f24-993c-23838f18d4ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:59.098768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c738995fef2342e99240de801574307d', 'request_id': 'req-9974c8cf-a299-4f24-993c-23838f18d4ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:59.206470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] The token cache is being invalidate because project c738995fef2342e99240de801574307d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:38:59.206695 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:59.207457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-9974c8cf-a299-4f24-993c-23838f18d4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:38:59.209278 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1254/2504] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:38:58 2022] DELETE /identity/v3/projects/c738995fef2342e99240de801574307d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:38:59.215134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86967c57-4c09-4989-b85c-f561c57fae15 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.215462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86967c57-4c09-4989-b85c-f561c57fae15 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.215765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86967c57-4c09-4989-b85c-f561c57fae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.271638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-86967c57-4c09-4989-b85c-f561c57fae15 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:59.272837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-86967c57-4c09-4989-b85c-f561c57fae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:59.331691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1251/2505] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:59.339498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968168-77af-4a1e-86c8-c1af80a70c79 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.339854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968168-77af-4a1e-86c8-c1af80a70c79 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.340138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968168-77af-4a1e-86c8-c1af80a70c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.360789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cf968168-77af-4a1e-86c8-c1af80a70c79 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:59.361873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cf968168-77af-4a1e-86c8-c1af80a70c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:59.370572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1255/2506] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:59.374661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86967c57-4c09-4989-b85c-f561c57fae15 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:59.384438 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86967c57-4c09-4989-b85c-f561c57fae15 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:59.388943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:59.389601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.389887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.390167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.390896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:38:59.391233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:59.392832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:59.404825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1252/2507] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:38:59.409073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968168-77af-4a1e-86c8-c1af80a70c79 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:59.419119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968168-77af-4a1e-86c8-c1af80a70c79 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:59.423896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:59.424539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.424828 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.425107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.425799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:59.428357 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:59.510904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1256/2508] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:38:59 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:38:59.515038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:59.525226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edee99b-ef53-47a7-91f9-1eb082b8a61d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:59.552802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:38:59.583670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:59.584339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.584687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.585165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.586431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:59.588402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:59.669387 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1253/2509] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:38:59 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:38:59.673393 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:59.684445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad163bf8-d84e-4f6e-b293-64526a210f30 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:59.689188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:59.689857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.690149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.690427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.691196 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:59.693535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:59.704184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1257/2510] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:38:59.708464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:59.719065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585deca-69b1-4797-8dee-8cf0e64a0e98 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:59.723931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619f94-012d-4770-8b7d-e480e492038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:59.724584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619f94-012d-4770-8b7d-e480e492038f admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.724868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619f94-012d-4770-8b7d-e480e492038f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.725152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619f94-012d-4770-8b7d-e480e492038f admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.725879 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619f94-012d-4770-8b7d-e480e492038f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:59.727749 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619f94-012d-4770-8b7d-e480e492038f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:59.738179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1254/2511] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:38:59.743456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:59.753399 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3947064-d4db-4f54-afd6-a4a8cc1e7b97 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:38:59.758619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:38:59.759536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.759821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.760101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] PATH_INFO: `/v3/projects/82044c0200ac407c92453318d4aafb5a/users/91c0ce85d4cd4660955287a8312dc2f5/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.775857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82044c0200ac407c92453318d4aafb5a, user_id=91c0ce85d4cd4660955287a8312dc2f5, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:38:59.778514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:38:59.853131 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1258/2512] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:38:59 2022] PUT /identity/v3/projects/82044c0200ac407c92453318d4aafb5a/users/91c0ce85d4cd4660955287a8312dc2f5/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:38:59.863549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:38:59.864370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:38:59.864652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:38:59.890835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] MFA Rules not processed for user `91c0ce85d4cd4660955287a8312dc2f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:38:59.892066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:38:59.957499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1255/2513] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:38:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:38:59.969596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:38:59.971391 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:39:00.056678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9686-d900-4e2f-8ce7-9343535b398c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:39:00.062229 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:39:00.062907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:39:00.063217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:39:00.063502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:39:00.071511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:39:00.073107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:39:00.081953 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1259/2514] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:38:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:39:09.954591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:39:10.077831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:39:45.421851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:39:45.423265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:39:45.479629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54b988-8d44-4ce7-9a2f-a7778feba2b1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:39:45.483340 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:39:45.483983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:39:45.484272 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:39:45.484554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:39:45.486423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:39:45.499995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:39:45.501412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:39:45.513041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1256/2515] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:39:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:39:55.510356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=26 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:39:55.564448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:39:55.574304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0444-1029-4629-a33d-84de53996c17 service nova] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:39:55.577965 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:39:55.578602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:39:55.578890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:39:55.579204 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:39:55.581112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:39:55.639879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:39:55.641315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:39:55.651444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1260/2516] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:39:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:40:05.646281 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=28 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:42.821953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:42.830632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc9d1-b59b-4a59-9e61-fa5b3a89927a service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:42.834235 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:42.834872 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:42.835160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:42.835465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:42.837200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:42.895432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:42.897010 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:42.906307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1257/2517] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:40:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:40:52.901808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=28 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.401778 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.403200 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:03.427062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9b979-396a-4908-b20d-b5d47beaaec2 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.431785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.432428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.432714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.432994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.434749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:03.494223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.495692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.504681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1261/2518] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:13.500617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=33 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:47.893845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:47.903415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf7895-5dbc-4bf0-b2c2-f22e4542b4e5 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:47.907072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:47.907751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.908041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.908321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.910138 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:47.973531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:47.975008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:47.984420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1258/2519] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:52.209613 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.210855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:52.274036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c6402-e2a2-4809-a781-bdae40c596a0 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.277677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c09a7-e890-49a5-a3d7-c998e6a79585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.278321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c09a7-e890-49a5-a3d7-c998e6a79585 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.278604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c09a7-e890-49a5-a3d7-c998e6a79585 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.278881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c09a7-e890-49a5-a3d7-c998e6a79585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.280683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a23c09a7-e890-49a5-a3d7-c998e6a79585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:52.322406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.323934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:52.347733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c09a7-e890-49a5-a3d7-c998e6a79585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.349831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c09a7-e890-49a5-a3d7-c998e6a79585 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.360242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1262/2520] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:52.392920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea91dd-10ba-4ff0-ac40-55e0b3d633ef service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.398099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.399011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.399336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.399620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] PATH_INFO: `/v3/users/c8f99704bcc0451bb08989609425dbcc` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.400831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f99704bcc0451bb08989609425dbcc)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.402770 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.653330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f99704bcc0451bb08989609425dbcc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:52.654034 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f99704bcc0451bb08989609425dbcc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:52.662723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f99704bcc0451bb08989609425dbcc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:52.663429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f99704bcc0451bb08989609425dbcc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:52.746805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f99704bcc0451bb08989609425dbcc'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:52.765826 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1259/2521] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:42:52 2022] DELETE /identity/v3/users/c8f99704bcc0451bb08989609425dbcc => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:42:52.770505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.770804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.771088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.790012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:52.791154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:52.856265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1263/2522] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:42:52.866916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.868451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:52.890378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:52.931883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae8538-0995-4809-9b8f-bd511f6e17d5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.935484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.936140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.936425 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.936704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.946892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.948363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.959938 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1260/2523] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:53.563625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:53.567092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:53.577924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a1be8-c09d-4b60-8c46-758e6d3a799f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:53.582728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:53.583716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.584034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.584321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] PATH_INFO: `/v3/projects/a692be969a3c4387933f0bc5349236c7` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.585516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a692be969a3c4387933f0bc5349236c7)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:53.587675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:53.736779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:53.739720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a692be969a3c4387933f0bc5349236c7', 'request_id': 'req-7822714e-7e13-463e-a0b0-6682e8b8e665'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:53.746016 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a692be969a3c4387933f0bc5349236c7', 'request_id': 'req-7822714e-7e13-463e-a0b0-6682e8b8e665'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:53.753465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a692be969a3c4387933f0bc5349236c7', 'request_id': 'req-7822714e-7e13-463e-a0b0-6682e8b8e665'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:53.850160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] The token cache is being invalidate because project a692be969a3c4387933f0bc5349236c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:42:53.850394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:53.852176 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:53.853762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1264/2524] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:42:53 2022] DELETE /identity/v3/projects/a692be969a3c4387933f0bc5349236c7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:55.970597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:55.972114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:56.047134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e255c-fabb-4ae0-996c-33685f590821 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.108132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.108913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.109139 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.109226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.111601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:56.115474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.117547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:56.128570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.130278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.141679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1261/2525] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:56.190564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822714e-7e13-463e-a0b0-6682e8b8e665 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.195319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.196213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.196478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.196760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] PATH_INFO: `/v3/users/fd3eef3ca9da4302a7c2e52acb8155c6` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.210086 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3eef3ca9da4302a7c2e52acb8155c6)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.212257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.403116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3eef3ca9da4302a7c2e52acb8155c6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:56.411931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3eef3ca9da4302a7c2e52acb8155c6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:56.412677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3eef3ca9da4302a7c2e52acb8155c6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:56.413925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3eef3ca9da4302a7c2e52acb8155c6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:56.426984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a9a8e3b-8bfb-480e-94e9-1436a91513b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3eef3ca9da4302a7c2e52acb8155c6'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:56.507668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1265/2526] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:42:56 2022] DELETE /identity/v3/users/fd3eef3ca9da4302a7c2e52acb8155c6 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:42:57.018464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.031098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d079d-9580-4e4e-a650-16f0f5c76a28 service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.084598 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=100 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:57.093688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.094558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.100225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.100225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] PATH_INFO: `/v3/projects/e07f9778bf194321ac9279121c065a3d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.106258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07f9778bf194321ac9279121c065a3d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.108407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.224952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07f9778bf194321ac9279121c065a3d', 'request_id': 'req-9f38e8cc-6b5d-4f6f-b608-4725185cb208'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:57.230768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07f9778bf194321ac9279121c065a3d', 'request_id': 'req-9f38e8cc-6b5d-4f6f-b608-4725185cb208'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:57.237706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07f9778bf194321ac9279121c065a3d', 'request_id': 'req-9f38e8cc-6b5d-4f6f-b608-4725185cb208'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:57.308329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] The token cache is being invalidate because project e07f9778bf194321ac9279121c065a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:42:57.308622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:57.309282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9f38e8cc-6b5d-4f6f-b608-4725185cb208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:57.310850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1262/2527] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:42:57 2022] DELETE /identity/v3/projects/e07f9778bf194321ac9279121c065a3d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:57.317119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc484d95-da7f-4ed1-9bd8-84eb9c8991ff None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.317606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc484d95-da7f-4ed1-9bd8-84eb9c8991ff None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.317752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc484d95-da7f-4ed1-9bd8-84eb9c8991ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.336599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-fc484d95-da7f-4ed1-9bd8-84eb9c8991ff None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:57.337823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-fc484d95-da7f-4ed1-9bd8-84eb9c8991ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:57.378226 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:57.400124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1266/2528] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:57.407737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f563f-3e26-47dc-8d4a-c4a713ae0f31 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.408136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f563f-3e26-47dc-8d4a-c4a713ae0f31 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.408367 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f563f-3e26-47dc-8d4a-c4a713ae0f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.427445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ee7f563f-3e26-47dc-8d4a-c4a713ae0f31 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:57.428672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ee7f563f-3e26-47dc-8d4a-c4a713ae0f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:57.437858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1263/2529] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:57.442882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc484d95-da7f-4ed1-9bd8-84eb9c8991ff None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.459320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc484d95-da7f-4ed1-9bd8-84eb9c8991ff None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.459320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.460104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.460582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.460877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.461620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:57.461937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.463587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.476587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1267/2530] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:57.481307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f563f-3e26-47dc-8d4a-c4a713ae0f31 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.505466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f563f-3e26-47dc-8d4a-c4a713ae0f31 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.505466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.505466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.505466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.505808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.505808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.505808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.628611 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1264/2531] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:42:57 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:42:57.632850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.642893 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da598f-3f42-49eb-bc95-c766c9025967 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.699457 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.700174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.700459 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.700739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.701435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.703382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.787022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1268/2532] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:42:57 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:57.791709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.801931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a44c5-372e-4e55-8225-92c840b2825e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.807135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.807817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.808103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.808382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.809150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.810733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.821025 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1265/2533] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:42:57.824867 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.835442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93d473-dbc0-45bc-b5fe-4ceea0d8edd6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.840034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6aea-4e23-402a-8335-9a535725fd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.840939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6aea-4e23-402a-8335-9a535725fd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.841225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6aea-4e23-402a-8335-9a535725fd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.841502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6aea-4e23-402a-8335-9a535725fd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.842228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e6aea-4e23-402a-8335-9a535725fd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.843866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e6aea-4e23-402a-8335-9a535725fd3d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.854260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1269/2534] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:42:57.858202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.868848 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d9019-cd9f-400a-894f-609aa5a3a372 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.873629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.874510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.874803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.875089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] PATH_INFO: `/v3/projects/a93338b6ded3448f80a166742333faff/users/3edba5dd77c1460ca759e5321e412181/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.890596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93338b6ded3448f80a166742333faff, user_id=3edba5dd77c1460ca759e5321e412181, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.893232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.999297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1266/2535] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:42:57 2022] PUT /identity/v3/projects/a93338b6ded3448f80a166742333faff/users/3edba5dd77c1460ca759e5321e412181/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:58.010732 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123581-5e79-4536-bac7-990b1f86d8b4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.011106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123581-5e79-4536-bac7-990b1f86d8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.011512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123581-5e79-4536-bac7-990b1f86d8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.037916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-d6123581-5e79-4536-bac7-990b1f86d8b4 None None] MFA Rules not processed for user `3edba5dd77c1460ca759e5321e412181`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:58.039754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-d6123581-5e79-4536-bac7-990b1f86d8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:58.104456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1270/2536] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:42:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:58.108853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.171640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3526-12ef-4dcd-a8b3-52738f574d0f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.176202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.176854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.177141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.177421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.178113 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.180035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.184013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:58.314614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1267/2537] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:42:58 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:42:58.318692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6123581-5e79-4536-bac7-990b1f86d8b4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.329987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6123581-5e79-4536-bac7-990b1f86d8b4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.362595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:58.384602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.385264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.385551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.385827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.386516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.388471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.523221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1271/2538] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:42:58 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:58.524672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.535567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f607b55-775c-4fd3-ac90-7d96639cd5c9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.541236 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.542327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.542618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.543056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.543825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.545470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.560318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1268/2539] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:42:58.561649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.572691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684b99a-a68d-41e5-911b-66578b7a73bf admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.577289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.578168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.578458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.578739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] PATH_INFO: `/v3/projects/8d2b437a52ed49819dfe5e50c8cd436a/users/54137a3a54e444b4a57ba9f9f0a6f3ce/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.594370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2b437a52ed49819dfe5e50c8cd436a, user_id=54137a3a54e444b4a57ba9f9f0a6f3ce, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.597127 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.696096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1272/2540] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:42:58 2022] PUT /identity/v3/projects/8d2b437a52ed49819dfe5e50c8cd436a/users/54137a3a54e444b4a57ba9f9f0a6f3ce/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:42:58.700082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.709993 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7563d9-de5d-46b2-9a97-48b85e3f5774 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.766714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d85b9-8f87-435c-9145-fdf1471efce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.767411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d85b9-8f87-435c-9145-fdf1471efce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.767701 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d85b9-8f87-435c-9145-fdf1471efce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.767984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d85b9-8f87-435c-9145-fdf1471efce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.768736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d85b9-8f87-435c-9145-fdf1471efce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d85b9-8f87-435c-9145-fdf1471efce8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.780563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1269/2541] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:42:58.784622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.795559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603da0c-4478-4cc1-b3c3-2e49d7782853 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.800183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.801066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.801354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.801637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] PATH_INFO: `/v3/projects/8d2b437a52ed49819dfe5e50c8cd436a/users/54137a3a54e444b4a57ba9f9f0a6f3ce/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.803655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2b437a52ed49819dfe5e50c8cd436a, user_id=54137a3a54e444b4a57ba9f9f0a6f3ce, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.806472 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.816575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1273/2542] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:42:58 2022] PUT /identity/v3/projects/8d2b437a52ed49819dfe5e50c8cd436a/users/54137a3a54e444b4a57ba9f9f0a6f3ce/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:42:58.826704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0ef27-b2c8-4c05-9ae3-f31be8af7ae3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.827002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0ef27-b2c8-4c05-9ae3-f31be8af7ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.827318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0ef27-b2c8-4c05-9ae3-f31be8af7ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.855710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-65d0ef27-b2c8-4c05-9ae3-f31be8af7ae3 None None] MFA Rules not processed for user `54137a3a54e444b4a57ba9f9f0a6f3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:58.856935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-65d0ef27-b2c8-4c05-9ae3-f31be8af7ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:58.925201 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1270/2543] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:42:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:58.936373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.938124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:58.996943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf47d1-31d7-4801-8918-9b03ea256ba3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.000556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393e912-bf4c-4c86-b7d9-830320d5fab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.001199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e912-bf4c-4c86-b7d9-830320d5fab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.001490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e912-bf4c-4c86-b7d9-830320d5fab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.001774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e912-bf4c-4c86-b7d9-830320d5fab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.011745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393e912-bf4c-4c86-b7d9-830320d5fab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.013180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393e912-bf4c-4c86-b7d9-830320d5fab1 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.024616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1274/2544] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:59.085354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0ef27-b2c8-4c05-9ae3-f31be8af7ae3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.146623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:59.151329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0ef27-b2c8-4c05-9ae3-f31be8af7ae3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.155897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.156779 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.157109 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.157353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] PATH_INFO: `/v3/users/3edba5dd77c1460ca759e5321e412181` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.158566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edba5dd77c1460ca759e5321e412181)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.160546 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.349083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edba5dd77c1460ca759e5321e412181'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:59.349735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edba5dd77c1460ca759e5321e412181'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:59.356571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edba5dd77c1460ca759e5321e412181'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:59.357193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edba5dd77c1460ca759e5321e412181'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:59.438381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edba5dd77c1460ca759e5321e412181'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:42:59.456293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1271/2545] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:42:59 2022] DELETE /identity/v3/users/3edba5dd77c1460ca759e5321e412181 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:59.460697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.461247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.461366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.479783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:59.480995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:59.543013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1275/2546] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:42:59.552599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.554072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:59.614166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7c3c2-4326-4c7d-a62c-c0da435821bc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.618626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.619317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.619607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.619888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.628919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.630358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.639769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1272/2547] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:42:59.972143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.973709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:59.987100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63cb11-4e2f-4c85-9b00-9eae9c1d94a9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.991743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.993169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.993451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.993823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] PATH_INFO: `/v3/projects/a93338b6ded3448f80a166742333faff` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.995471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93338b6ded3448f80a166742333faff)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.997404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:00.081114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:00.096300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93338b6ded3448f80a166742333faff', 'request_id': 'req-b633930c-9aa2-44fa-a7a1-dc2865fed770'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.102804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93338b6ded3448f80a166742333faff', 'request_id': 'req-b633930c-9aa2-44fa-a7a1-dc2865fed770'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.109345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93338b6ded3448f80a166742333faff', 'request_id': 'req-b633930c-9aa2-44fa-a7a1-dc2865fed770'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.192593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] The token cache is being invalidate because project a93338b6ded3448f80a166742333faff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:00.192964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.193592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.195116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1276/2548] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:42:59 2022] DELETE /identity/v3/projects/a93338b6ded3448f80a166742333faff => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:00.203538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:00.213805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4838c-14bc-4596-aa28-0baccbf28b92 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:00.270852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:00.271766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:00.272055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:00.272336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] PATH_INFO: `/v3/users/54137a3a54e444b4a57ba9f9f0a6f3ce` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:00.274151 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54137a3a54e444b4a57ba9f9f0a6f3ce)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:00.276575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:00.298589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:00.513605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54137a3a54e444b4a57ba9f9f0a6f3ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.514205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54137a3a54e444b4a57ba9f9f0a6f3ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.521037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54137a3a54e444b4a57ba9f9f0a6f3ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.521713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54137a3a54e444b4a57ba9f9f0a6f3ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.603630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d75e5400-0882-4661-9c6a-a77156873fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54137a3a54e444b4a57ba9f9f0a6f3ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:00.623108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1273/2549] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:00 2022] DELETE /identity/v3/users/54137a3a54e444b4a57ba9f9f0a6f3ce => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:01.040311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.041791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:01.055825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633930c-9aa2-44fa-a7a1-dc2865fed770 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:01.114038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:01.115067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.115484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.115745 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] PATH_INFO: `/v3/projects/8d2b437a52ed49819dfe5e50c8cd436a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.116982 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2b437a52ed49819dfe5e50c8cd436a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:01.120098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:01.217089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2b437a52ed49819dfe5e50c8cd436a', 'request_id': 'req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:01.222604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2b437a52ed49819dfe5e50c8cd436a', 'request_id': 'req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:01.230079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2b437a52ed49819dfe5e50c8cd436a', 'request_id': 'req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:01.300910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] The token cache is being invalidate because project 8d2b437a52ed49819dfe5e50c8cd436a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:01.301135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:01.301777 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8f47dc2-5ff4-4382-8a7b-5bdc9e29fe4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:01.303348 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1277/2550] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:01 2022] DELETE /identity/v3/projects/8d2b437a52ed49819dfe5e50c8cd436a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:01.308962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996c4c1-fc91-458f-bdb7-603b3b8c9cca None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.309404 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996c4c1-fc91-458f-bdb7-603b3b8c9cca None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.309586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996c4c1-fc91-458f-bdb7-603b3b8c9cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.327902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7996c4c1-fc91-458f-bdb7-603b3b8c9cca None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:01.329140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7996c4c1-fc91-458f-bdb7-603b3b8c9cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:01.387784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1274/2551] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:01.395424 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96f9f-ab03-4b41-af49-e8fa88c55ae2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.395727 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96f9f-ab03-4b41-af49-e8fa88c55ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.396013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96f9f-ab03-4b41-af49-e8fa88c55ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.413780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ecd96f9f-ab03-4b41-af49-e8fa88c55ae2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:01.415749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ecd96f9f-ab03-4b41-af49-e8fa88c55ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:01.423579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1278/2552] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:01.427537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996c4c1-fc91-458f-bdb7-603b3b8c9cca None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.437818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996c4c1-fc91-458f-bdb7-603b3b8c9cca None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:01.442382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:01.443028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.443359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.443640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.444363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:01.444695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:01.446316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:01.457858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1275/2553] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:01.462660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd96f9f-ab03-4b41-af49-e8fa88c55ae2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.472402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd96f9f-ab03-4b41-af49-e8fa88c55ae2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:01.477042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:01.478421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.478708 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.478994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.479718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:01.481624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:01.600464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1279/2554] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:01 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:01.605225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.615102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d40171-0193-45c2-959e-8793f92605ec admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:01.677691 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:01.678384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.678672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.678982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.679715 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:01.681997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:01.770216 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1276/2555] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:01 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:01.774784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.785154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc3262-e362-4e3c-b51e-76cae6ff091c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:01.790567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:01.791465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.791748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.792024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.792754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:01.794342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:01.805994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1280/2556] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:01.809814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.819843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538c5b5-e10a-4905-96f9-ab2bacc9bd97 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:01.824599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8e275-8b25-47d7-b3f9-449a295cfbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:01.825274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8e275-8b25-47d7-b3f9-449a295cfbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.825565 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8e275-8b25-47d7-b3f9-449a295cfbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.825890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8e275-8b25-47d7-b3f9-449a295cfbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.826616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8e275-8b25-47d7-b3f9-449a295cfbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:01.828239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8e275-8b25-47d7-b3f9-449a295cfbe5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:01.839208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1277/2557] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:01.843106 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.853907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91308c3-a18f-4bfa-a125-a38cf3b24017 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:01.858571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:01.859482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.859772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.860067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] PATH_INFO: `/v3/projects/d2ec007bcd2e480aab33c6d248e85b35/users/c1f4a303891f4dde9b99171474b1f6ba/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.862304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:01.875013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ec007bcd2e480aab33c6d248e85b35, user_id=c1f4a303891f4dde9b99171474b1f6ba, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:01.877822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:01.954105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1281/2558] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:01 2022] PUT /identity/v3/projects/d2ec007bcd2e480aab33c6d248e85b35/users/c1f4a303891f4dde9b99171474b1f6ba/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:01.966465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a8b5c-320f-4f83-8ad3-cf6ee69cbab2 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.966764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a8b5c-320f-4f83-8ad3-cf6ee69cbab2 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.967050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a8b5c-320f-4f83-8ad3-cf6ee69cbab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.974255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:01.994445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7d6a8b5c-320f-4f83-8ad3-cf6ee69cbab2 None None] MFA Rules not processed for user `c1f4a303891f4dde9b99171474b1f6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:01.995701 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7d6a8b5c-320f-4f83-8ad3-cf6ee69cbab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:02.060461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1278/2559] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:02.064823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.128058 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a7633-70a4-4256-981f-a23046b05e33 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.132628 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.133764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.134090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.134371 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.135096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.137041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.223883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1282/2560] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:02 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:02.228280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a8b5c-320f-4f83-8ad3-cf6ee69cbab2 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.239102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a8b5c-320f-4f83-8ad3-cf6ee69cbab2 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.301005 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.301683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.301974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.302261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.302951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.304914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.388016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1279/2561] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:02 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:02.391825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.402235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f313ea1-61d3-4e6f-9836-74db49f15c86 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.406749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.407729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.408013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.408287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.409009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.410609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.421428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1283/2562] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:02.425190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.435680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c5e70-b29a-4dac-b076-62a24fb67292 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.440275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01f6c-4f99-4c85-a1ab-220e5460e4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.440920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01f6c-4f99-4c85-a1ab-220e5460e4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.441209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01f6c-4f99-4c85-a1ab-220e5460e4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.441491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01f6c-4f99-4c85-a1ab-220e5460e4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.442220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01f6c-4f99-4c85-a1ab-220e5460e4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.443859 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01f6c-4f99-4c85-a1ab-220e5460e4cd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.454873 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1280/2563] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:02.458827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.469285 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd1d08-391f-466e-913f-848818b9ad32 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.473922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.474792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.475081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.475398 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] PATH_INFO: `/v3/projects/b4392bf617384a90ac8310068f51db30/users/2fb744f6f5544471a1f175cf3d033c27/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.489555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4392bf617384a90ac8310068f51db30, user_id=2fb744f6f5544471a1f175cf3d033c27, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.492341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.559877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1284/2564] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:02 2022] PUT /identity/v3/projects/b4392bf617384a90ac8310068f51db30/users/2fb744f6f5544471a1f175cf3d033c27/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:02.570115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e16c4-7558-469a-a62e-6430699f9aa3 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.570413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e16c4-7558-469a-a62e-6430699f9aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.570696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e16c4-7558-469a-a62e-6430699f9aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.598567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-178e16c4-7558-469a-a62e-6430699f9aa3 None None] MFA Rules not processed for user `2fb744f6f5544471a1f175cf3d033c27`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:02.599830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-178e16c4-7558-469a-a62e-6430699f9aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:02.667499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1281/2565] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:02.671135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.735596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d73427-814e-4140-9b25-c298dde13369 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.740416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.741076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.741364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.741647 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.742337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.745066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.834382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1285/2566] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:02 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:02.838659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e16c4-7558-469a-a62e-6430699f9aa3 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.844699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.850489 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e16c4-7558-469a-a62e-6430699f9aa3 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.907161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.907807 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.908075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.908352 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.909040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.910952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.998435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1282/2567] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:02 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:03.002460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.012895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24aa25-0f22-4113-8164-0549e5af07ed admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.017559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.018217 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.018503 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.018786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.019542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.021135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.031986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1286/2568] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:03.035992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.046767 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bcaca-f0b8-4d7c-95a7-08f96becd28a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.051378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.074865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.074865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.074865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] PATH_INFO: `/v3/projects/a0e621fbe655402f8245279af6012ebb/users/3c328c1b26244d9296d78cec29834f03/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.074865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e621fbe655402f8245279af6012ebb, user_id=3c328c1b26244d9296d78cec29834f03, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.074865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.165203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1283/2569] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:03 2022] PUT /identity/v3/projects/a0e621fbe655402f8245279af6012ebb/users/3c328c1b26244d9296d78cec29834f03/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:03.169579 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.179765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcc617-1ef0-4eb8-8fa5-2e317567d8e3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.183974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.236844 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c4fa8-7224-4995-8b59-0a34b15644da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.237740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c4fa8-7224-4995-8b59-0a34b15644da admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.238429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c4fa8-7224-4995-8b59-0a34b15644da admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.238756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c4fa8-7224-4995-8b59-0a34b15644da admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.239551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c4fa8-7224-4995-8b59-0a34b15644da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.241201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c4fa8-7224-4995-8b59-0a34b15644da admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.251072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1287/2570] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:03.255218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.265660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78064bab-0108-46f8-9f5d-8947f82c95ba admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.270339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.271239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.271531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.271813 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] PATH_INFO: `/v3/projects/a0e621fbe655402f8245279af6012ebb/users/3c328c1b26244d9296d78cec29834f03/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.273781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e621fbe655402f8245279af6012ebb, user_id=3c328c1b26244d9296d78cec29834f03, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.276978 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.287106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1284/2571] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:03 2022] PUT /identity/v3/projects/a0e621fbe655402f8245279af6012ebb/users/3c328c1b26244d9296d78cec29834f03/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:03.298972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a1796-30ff-4ba1-89d3-29cd82390c6d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.299298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a1796-30ff-4ba1-89d3-29cd82390c6d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.299581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a1796-30ff-4ba1-89d3-29cd82390c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.326093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-349a1796-30ff-4ba1-89d3-29cd82390c6d None None] MFA Rules not processed for user `3c328c1b26244d9296d78cec29834f03`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:03.327330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-349a1796-30ff-4ba1-89d3-29cd82390c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:03.396155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1288/2572] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:03.407801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.409642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:03.466857 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34b71a-8fef-4323-ade4-80cae804e74f admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.470458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.471103 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.471432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.471718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.480818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.482246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.491866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1285/2573] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:03.628890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a1796-30ff-4ba1-89d3-29cd82390c6d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.639084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a1796-30ff-4ba1-89d3-29cd82390c6d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.642654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.643334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.643623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.643907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.699230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.700662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.709924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1289/2574] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:13.487654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:13.705383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:31.626601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:31.626601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:31.644345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:31.646394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:31.723881 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc13895-8fb9-4ab3-97e1-9dd86d5645e6 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:31.732291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:31.732291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:31.732291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:31.732291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:31.732627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:31.736142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a298ec1-83b9-457b-ab64-16cc9431e0f7 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:31.740585 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:31.742833 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:31.743415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:31.743415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:31.745456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] Could not find project: bb07d6b829814af5af6afa53a1e4a3ad.: keystone.exception.ProjectNotFound: Could not find project: bb07d6b829814af5af6afa53a1e4a3ad. Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jun 23 11:43:31.746766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:31.747909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self._validate_token_resources() Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:31.749035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:43:31.750556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:43:31.752075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: bb07d6b829814af5af6afa53a1e4a3ad. Jun 23 11:43:31.752075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: ERROR keystone.server.flask.application Jun 23 11:43:31.752075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1286/2575] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:31 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 130 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] Could not find project: 53f26e0accb54e6b85adfa73de12b3af.: keystone.exception.ProjectNotFound: Could not find project: 53f26e0accb54e6b85adfa73de12b3af. Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jun 23 11:43:31.770330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:31.771441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self._validate_token_resources() Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:31.772980 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:43:31.773963 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:43:31.775139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 53f26e0accb54e6b85adfa73de12b3af. Jun 23 11:43:31.775139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: ERROR keystone.server.flask.application Jun 23 11:43:31.775139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1290/2576] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:31 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 129 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Jun 23 11:43:31.918012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:31.919441 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:31.940209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c308b-e5f1-45a2-b784-674c67ea9919 service glance] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:31.944921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:31.945693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:31.945940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:31.946258 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:31.948391 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:32.020017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:32.021523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:32.031887 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1287/2577] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:43:32.396706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:32.408304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e0c09-4f02-4989-a74f-27a0f92baa8f service glance] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:32.412255 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b875fa-0f7c-405d-9008-414c337f74d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:32.413576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b875fa-0f7c-405d-9008-414c337f74d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:32.414073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b875fa-0f7c-405d-9008-414c337f74d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:32.414431 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b875fa-0f7c-405d-9008-414c337f74d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:32.416279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-06b875fa-0f7c-405d-9008-414c337f74d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:32.479096 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b875fa-0f7c-405d-9008-414c337f74d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:32.480506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b875fa-0f7c-405d-9008-414c337f74d0 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:32.489583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1291/2578] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:36.055167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:36.121814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13845e5-2cd8-4182-812f-9ac8617620d7 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:36.126355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:36.127266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:36.127554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:36.127835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] PATH_INFO: `/v3/users/c1f4a303891f4dde9b99171474b1f6ba` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:36.129008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f4a303891f4dde9b99171474b1f6ba)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:36.130944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:36.156692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:36.387435 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f4a303891f4dde9b99171474b1f6ba'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.388095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f4a303891f4dde9b99171474b1f6ba'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.394804 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f4a303891f4dde9b99171474b1f6ba'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.395464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f4a303891f4dde9b99171474b1f6ba'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.459901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f4a303891f4dde9b99171474b1f6ba'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.479645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1288/2579] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:36 2022] DELETE /identity/v3/users/c1f4a303891f4dde9b99171474b1f6ba => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:36.484084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:36.484558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:36.484781 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:36.508811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:36.511728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:36.511828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:36.512345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:36.578068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1292/2580] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:36.582936 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2221a-d776-44f3-a509-728292604472 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:36.587875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:36.588351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:36.589012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:36.589296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:36.589575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] PATH_INFO: `/v3/users/91c0ce85d4cd4660955287a8312dc2f5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:36.590015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:36.590911 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c0ce85d4cd4660955287a8312dc2f5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:36.592890 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:36.643689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:36.649208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538f5f7-c29a-4704-862f-d90ab2a57148 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:36.653032 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040048a-90bf-473b-b180-435428168682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:36.653685 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040048a-90bf-473b-b180-435428168682 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:36.653975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040048a-90bf-473b-b180-435428168682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:36.654256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040048a-90bf-473b-b180-435428168682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:36.663478 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040048a-90bf-473b-b180-435428168682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:36.665730 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040048a-90bf-473b-b180-435428168682 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:36.675241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1293/2581] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:43:36.779667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c0ce85d4cd4660955287a8312dc2f5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.780429 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c0ce85d4cd4660955287a8312dc2f5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.787953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c0ce85d4cd4660955287a8312dc2f5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.788629 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c0ce85d4cd4660955287a8312dc2f5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.882920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c0ce85d4cd4660955287a8312dc2f5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:36.909128 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1289/2582] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:36 2022] DELETE /identity/v3/users/91c0ce85d4cd4660955287a8312dc2f5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:36.916702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:36.916702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:36.916702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:36.932876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:36.934024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:36.996505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1294/2583] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:37.006163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:37.007964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:37.030897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:37.072769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615e7a-04fd-471a-aa77-3f87c289b846 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:37.073824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:37.075386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:37.077371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:37.083412 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:37.087360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:37.088972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:37.090693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccaa51-437a-49ec-89f0-f2e2dd7545d6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:37.096238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:37.097239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:37.097678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:37.097988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] PATH_INFO: `/v3/projects/d2ec007bcd2e480aab33c6d248e85b35` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:37.099423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ec007bcd2e480aab33c6d248e85b35)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:37.102811 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:37.111408 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:37.112918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:37.124628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1290/2584] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:37.198724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ec007bcd2e480aab33c6d248e85b35', 'request_id': 'req-b399c366-675e-4073-ad97-7adde0eb8c1a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.207007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ec007bcd2e480aab33c6d248e85b35', 'request_id': 'req-b399c366-675e-4073-ad97-7adde0eb8c1a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.215088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ec007bcd2e480aab33c6d248e85b35', 'request_id': 'req-b399c366-675e-4073-ad97-7adde0eb8c1a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.328350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] The token cache is being invalidate because project d2ec007bcd2e480aab33c6d248e85b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:37.328653 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.329340 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.331062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1295/2585] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:37 2022] DELETE /identity/v3/projects/d2ec007bcd2e480aab33c6d248e85b35 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:37.335042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:37.346337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587b847-5e75-4814-aee5-a1558ef0ae7b service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:37.410846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:37.411761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:37.412124 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:37.412473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] PATH_INFO: `/v3/users/2fb744f6f5544471a1f175cf3d033c27` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:37.413963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb744f6f5544471a1f175cf3d033c27)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:37.415967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:37.569360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:37.571920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:37.579564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb744f6f5544471a1f175cf3d033c27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.580283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb744f6f5544471a1f175cf3d033c27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.590092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399c366-675e-4073-ad97-7adde0eb8c1a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:37.595300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:37.596276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:37.596689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:37.597010 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] PATH_INFO: `/v3/projects/82044c0200ac407c92453318d4aafb5a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:37.598331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82044c0200ac407c92453318d4aafb5a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:37.600310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:37.600682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb744f6f5544471a1f175cf3d033c27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.601906 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb744f6f5544471a1f175cf3d033c27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.659831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-5f92ebe1-0e6c-4c70-9633-62ce2cc3922f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb744f6f5544471a1f175cf3d033c27'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.677842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1291/2586] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:37 2022] DELETE /identity/v3/users/2fb744f6f5544471a1f175cf3d033c27 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:37.715009 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82044c0200ac407c92453318d4aafb5a', 'request_id': 'req-a8ad1a8c-8ecc-4795-9539-752cf13f7471'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.723113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82044c0200ac407c92453318d4aafb5a', 'request_id': 'req-a8ad1a8c-8ecc-4795-9539-752cf13f7471'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.730974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82044c0200ac407c92453318d4aafb5a', 'request_id': 'req-a8ad1a8c-8ecc-4795-9539-752cf13f7471'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.735238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:37.802538 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] The token cache is being invalidate because project 82044c0200ac407c92453318d4aafb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:37.802916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.803651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a8ad1a8c-8ecc-4795-9539-752cf13f7471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:37.806121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1296/2587] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:37 2022] DELETE /identity/v3/projects/82044c0200ac407c92453318d4aafb5a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:37.810746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e53b-007e-47d3-8da1-7229f08eb640 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:37.811048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e53b-007e-47d3-8da1-7229f08eb640 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:37.811365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e53b-007e-47d3-8da1-7229f08eb640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:37.832173 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3b00e53b-007e-47d3-8da1-7229f08eb640 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:37.833307 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3b00e53b-007e-47d3-8da1-7229f08eb640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:37.874816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:37.904632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1292/2588] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:37.919892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1a768-1343-4fb7-9c67-964d51634fa6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:37.919892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1a768-1343-4fb7-9c67-964d51634fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:37.919892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1a768-1343-4fb7-9c67-964d51634fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:37.939506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-08b1a768-1343-4fb7-9c67-964d51634fa6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:37.940823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-08b1a768-1343-4fb7-9c67-964d51634fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:37.959335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1297/2589] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:37.959710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e53b-007e-47d3-8da1-7229f08eb640 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:37.978071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e53b-007e-47d3-8da1-7229f08eb640 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:37.983192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:37.984003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:37.984191 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:37.984477 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:37.985214 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:37.985659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:37.987390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.000165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1293/2590] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:38.004611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1a768-1343-4fb7-9c67-964d51634fa6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.014594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1a768-1343-4fb7-9c67-964d51634fa6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.019347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.020013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.020302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.021502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.021502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.023542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.113825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1298/2591] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:38 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:38.119066 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.131960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f83228-0d9f-4401-8dc9-644c12d4e3d1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.138468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.141160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:38.199712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.200455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.209155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.209155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.209155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.209155 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.225257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83153b-0af8-4a9a-8893-7f26ae3d44ca admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.231330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.232191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.232476 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.232753 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] PATH_INFO: `/v3/projects/b4392bf617384a90ac8310068f51db30` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.233994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4392bf617384a90ac8310068f51db30)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.236087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.325945 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1294/2592] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:38 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:38.326973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.337505 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525af62b-17a6-4f88-a7ed-b031d3052a94 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.342491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.343169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.343509 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.343798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.344531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.346137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.367992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1295/2593] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:43:38.370638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.379319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4392bf617384a90ac8310068f51db30', 'request_id': 'req-7df180ab-b300-48f3-a697-7292d0199fb2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.381205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfb071-4fdf-410a-bb09-74d53596b1a4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.386370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4392bf617384a90ac8310068f51db30', 'request_id': 'req-7df180ab-b300-48f3-a697-7292d0199fb2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.392824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4392bf617384a90ac8310068f51db30', 'request_id': 'req-7df180ab-b300-48f3-a697-7292d0199fb2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.439914 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.440618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.440907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.441188 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.441917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.443616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.456107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1296/2594] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:38.460189 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.470763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc3bfc-f135-4f06-9ba5-9c34e68ec61b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.475457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f5efa-f4f1-4c6c-b1a5-47c160b3d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.476968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5efa-f4f1-4c6c-b1a5-47c160b3d896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.477252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5efa-f4f1-4c6c-b1a5-47c160b3d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.477532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5efa-f4f1-4c6c-b1a5-47c160b3d896 admin admin] PATH_INFO: `/v3/projects/de76c80ac72c46c5a17755d6ac45a199/users/8b80f488fb5d46f99df9845823280224/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.492860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f5efa-f4f1-4c6c-b1a5-47c160b3d896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de76c80ac72c46c5a17755d6ac45a199, user_id=8b80f488fb5d46f99df9845823280224, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.496559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] The token cache is being invalidate because project b4392bf617384a90ac8310068f51db30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:38.496559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.496559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.496803 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f5efa-f4f1-4c6c-b1a5-47c160b3d896 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.496889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1299/2595] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:38 2022] DELETE /identity/v3/projects/b4392bf617384a90ac8310068f51db30 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:38.499655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.501110 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:38.513645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df180ab-b300-48f3-a697-7292d0199fb2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.518423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.519315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.519604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.519886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] PATH_INFO: `/v3/users/3c328c1b26244d9296d78cec29834f03` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.521080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c328c1b26244d9296d78cec29834f03)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.523590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.580658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1297/2596] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:38 2022] PUT /identity/v3/projects/de76c80ac72c46c5a17755d6ac45a199/users/8b80f488fb5d46f99df9845823280224/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:38.593271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.593667 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.594004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.620814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] MFA Rules not processed for user `8b80f488fb5d46f99df9845823280224`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:38.621959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:38.638486 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:38.691998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1298/2597] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:38.697259 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.698606 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:38.708754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c328c1b26244d9296d78cec29834f03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.714786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:38.718803 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c328c1b26244d9296d78cec29834f03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.719591 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c328c1b26244d9296d78cec29834f03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.720286 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c328c1b26244d9296d78cec29834f03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.732977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c328c1b26244d9296d78cec29834f03'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:38.764910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681be66-3be0-4d07-a2c4-3f4a2e0252dc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.769554 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.770212 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.770498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.770778 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.771497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.774020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.819363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1300/2598] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:38 2022] DELETE /identity/v3/users/3c328c1b26244d9296d78cec29834f03 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:38.896147 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1299/2599] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:38 2022] POST /identity/v3/projects => generated 371 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:38.900736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.902190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:38.968866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9b08b-7d70-4552-b787-b031149eacfd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.026035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.026840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.027125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.027443 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.028137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.030785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.107615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1301/2600] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:38 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:39.111523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.122982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75977c51-3bae-4410-9e40-38c4dc34f899 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.127739 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.128463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.128819 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.129108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.129842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.131538 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.143396 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1300/2601] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:39.147543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.158415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f19b3-8ccf-4683-9339-df333ea73d8e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.163205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.164254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.164541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.164822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] PATH_INFO: `/v3/projects/59fd8eeb12dc4656929280a79d0a3d5d/users/166e010c481e49a1b202704eafe2521c/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.183780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fd8eeb12dc4656929280a79d0a3d5d, user_id=166e010c481e49a1b202704eafe2521c, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.186831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.295627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.297462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:39.302678 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1302/2602] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:39 2022] PUT /identity/v3/projects/59fd8eeb12dc4656929280a79d0a3d5d/users/166e010c481e49a1b202704eafe2521c/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:39.303760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.328984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6372944-25d0-4aba-a520-e14fbf59ee38 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.368081 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:39.386694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8fbab-e154-40c0-bb7a-4788ba85b10d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.391660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.392325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.392612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.392896 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.393629 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.395262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.399627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.400490 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.400780 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.401078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] PATH_INFO: `/v3/projects/a0e621fbe655402f8245279af6012ebb` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.402275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e621fbe655402f8245279af6012ebb)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.404230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.405826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1303/2603] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:39.409531 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.419437 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fae916-d058-40fd-8980-7361bfe7c89e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.424055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488aebe-2cbf-48a3-8702-f117b338292f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.424922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488aebe-2cbf-48a3-8702-f117b338292f admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.425205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488aebe-2cbf-48a3-8702-f117b338292f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.425480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488aebe-2cbf-48a3-8702-f117b338292f admin admin] PATH_INFO: `/v3/projects/59fd8eeb12dc4656929280a79d0a3d5d/users/166e010c481e49a1b202704eafe2521c/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.427740 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488aebe-2cbf-48a3-8702-f117b338292f admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fd8eeb12dc4656929280a79d0a3d5d, user_id=166e010c481e49a1b202704eafe2521c, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.430941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488aebe-2cbf-48a3-8702-f117b338292f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.442147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1304/2604] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:39 2022] PUT /identity/v3/projects/59fd8eeb12dc4656929280a79d0a3d5d/users/166e010c481e49a1b202704eafe2521c/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:39.453076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.453448 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.453730 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.479768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] MFA Rules not processed for user `166e010c481e49a1b202704eafe2521c`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:39.480971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:39.518092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:39.537132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e621fbe655402f8245279af6012ebb', 'request_id': 'req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:39.547108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e621fbe655402f8245279af6012ebb', 'request_id': 'req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:39.548527 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1305/2605] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:39.559508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e621fbe655402f8245279af6012ebb', 'request_id': 'req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:39.561887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.565688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:39.618723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6481ef6-26ae-4c96-b83b-6f6acda93be3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.665011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.665589 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.665876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.666187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.712903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:39.718153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] The token cache is being invalidate because project a0e621fbe655402f8245279af6012ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:39.720403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:39.720403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3a63862e-db9e-4a8e-9bfb-b92cb9c0512b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:39.720835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1301/2606] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:39 2022] DELETE /identity/v3/projects/a0e621fbe655402f8245279af6012ebb => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:39.727325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ece15-9591-4609-857c-fce4f401661c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.727549 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ece15-9591-4609-857c-fce4f401661c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.727828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ece15-9591-4609-857c-fce4f401661c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.730222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.731801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.742365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1306/2607] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:43:39.750319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1a5ece15-9591-4609-857c-fce4f401661c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:39.751578 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1a5ece15-9591-4609-857c-fce4f401661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:39.808258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.809681 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:39.831835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1302/2608] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:39.841510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c744351-aa3e-4643-99ed-91e95ba9aced None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.842050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c744351-aa3e-4643-99ed-91e95ba9aced None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.842344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c744351-aa3e-4643-99ed-91e95ba9aced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.870827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1c744351-aa3e-4643-99ed-91e95ba9aced None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:39.872206 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1c744351-aa3e-4643-99ed-91e95ba9aced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:39.883694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1303/2609] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:39.884591 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c744351-aa3e-4643-99ed-91e95ba9aced None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.893796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c744351-aa3e-4643-99ed-91e95ba9aced None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.897103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1938cf-bfe3-4b99-b108-e56cbae87cf8 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.899227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.899897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.900186 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.900581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.901309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:39.901623 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.903379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.903524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.904516 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.904905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.905287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] PATH_INFO: `/v3/users/e40675ea75c04ad297d47623d2f60511` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.906632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40675ea75c04ad297d47623d2f60511)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.908974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.924851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1304/2610] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:39.925920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.937096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8679f-da87-4ef2-8e11-74396a4a7e56 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.942992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.943734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.944024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.944314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.945071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.947051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.071962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1305/2611] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:39 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:40.071962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:40.071962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:40.126143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975a8ba-4de9-4d14-aedf-e7cc1fc300b4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:40.129972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:40.130616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.130916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.131200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.133671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:40.161659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:40.181151 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40675ea75c04ad297d47623d2f60511'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:40.188112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40675ea75c04ad297d47623d2f60511'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:40.188764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40675ea75c04ad297d47623d2f60511'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:40.189374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40675ea75c04ad297d47623d2f60511'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:40.193370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:40.195522 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.201774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-46679235-ed3e-468d-ac6c-9adc27a5e9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40675ea75c04ad297d47623d2f60511'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:40.204613 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1306/2612] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:40.205386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:40.273823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6094b1-7d49-4984-99d0-1b7afbb19cae service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:40.278860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:40.279561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.279856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.280140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.280880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:40.282007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1307/2613] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:39 2022] DELETE /identity/v3/users/e40675ea75c04ad297d47623d2f60511 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:40.282799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.286482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.286772 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.287062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.304877 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:40.306157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:40.338296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:40.367565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1308/2614] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:40.370944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1307/2615] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:40 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:40.372810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:40.374267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:40.377045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:40.378659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:40.386948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c52ce-9203-432c-9bf9-26c573ef8de6 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:40.391900 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:40.392599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.392884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.393164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.393950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:40.395795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.406471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1309/2616] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:43:40.410363 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:40.419922 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521667d-7c54-491b-9444-ef53ea5868b5 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:40.424543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:40.425195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.425479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.425749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.426471 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:40.428076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.438693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1310/2617] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:43:40.441544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1175811-7f71-4d86-9bb0-4807b70e584c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:40.442839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:40.453145 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e77c5-4890-4b35-9fb4-7d9200ee9745 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:40.462859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:40.464364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.464506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.464720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] PATH_INFO: `/v3/projects/5e01c4c716ee4d128955d5a21b328155/users/cb6e4992e2d04aee8eed8f2c7314568c/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.480990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e01c4c716ee4d128955d5a21b328155, user_id=cb6e4992e2d04aee8eed8f2c7314568c, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:40.483874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.493631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0832251-a9ff-4f2c-ba04-84ad6031176f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:40.494292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0832251-a9ff-4f2c-ba04-84ad6031176f service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.494586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0832251-a9ff-4f2c-ba04-84ad6031176f service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.494874 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0832251-a9ff-4f2c-ba04-84ad6031176f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.504513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0832251-a9ff-4f2c-ba04-84ad6031176f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:40.506004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0832251-a9ff-4f2c-ba04-84ad6031176f service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.516246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1308/2618] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:40.544850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1311/2619] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:40 2022] PUT /identity/v3/projects/5e01c4c716ee4d128955d5a21b328155/users/cb6e4992e2d04aee8eed8f2c7314568c/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:40.550321 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.550619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.550902 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.577744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:40.579014 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:40.589458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:40.640277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1309/2620] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:40.649816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:40.705595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aeac2d-f658-4e68-8cba-968a2f65eea4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:40.709484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:40.710238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:40.710707 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:40.711161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:40.723362 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:40.726804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:40.738198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1312/2621] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:41.112661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:41.117903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:41.128215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04cd52-46e9-4e01-afef-23aea9b4c0dc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:41.148840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:41.148840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:41.148840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:41.148840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] PATH_INFO: `/v3/projects/bffdd5a183af4c379f7f86f46f39b70b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:41.150500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bffdd5a183af4c379f7f86f46f39b70b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:41.152528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:41.293382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bffdd5a183af4c379f7f86f46f39b70b', 'request_id': 'req-20dd6762-b328-41c5-bcfc-dae6d2090d62'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.300452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bffdd5a183af4c379f7f86f46f39b70b', 'request_id': 'req-20dd6762-b328-41c5-bcfc-dae6d2090d62'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.314008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bffdd5a183af4c379f7f86f46f39b70b', 'request_id': 'req-20dd6762-b328-41c5-bcfc-dae6d2090d62'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.416496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] The token cache is being invalidate because project bffdd5a183af4c379f7f86f46f39b70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:41.416799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.417513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.419190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1310/2622] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:41 2022] DELETE /identity/v3/projects/bffdd5a183af4c379f7f86f46f39b70b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:41.423469 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:41.436035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5febdb8-e0aa-471f-9a81-900503551b23 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:41.502625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:41.503662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:41.504390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:41.504390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] PATH_INFO: `/v3/users/336995499cb642c7b8869b120f83b68e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:41.505423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] RBAC: Authorizing `identity:delete_user(user_id=336995499cb642c7b8869b120f83b68e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:41.507522 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:41.695816 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:41.706951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336995499cb642c7b8869b120f83b68e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.716120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336995499cb642c7b8869b120f83b68e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.716885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336995499cb642c7b8869b120f83b68e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.717554 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336995499cb642c7b8869b120f83b68e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.731826 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-ab133849-72f5-4c0e-836d-8428a80b318b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336995499cb642c7b8869b120f83b68e'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:41.837643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1313/2623] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:41 2022] DELETE /identity/v3/users/336995499cb642c7b8869b120f83b68e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:42.227058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:42.228563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:42.259944 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd6762-b328-41c5-bcfc-dae6d2090d62 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:42.261756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:42.262107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:42.262573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:42.313164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] MFA Rules not processed for user `cb6e4992e2d04aee8eed8f2c7314568c`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:42.314735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:42.337370 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:42.338257 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:42.338806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:42.339098 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] PATH_INFO: `/v3/projects/260b96e94c394d8b871cc592da906fbf` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:42.342718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=260b96e94c394d8b871cc592da906fbf)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:42.342718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:42.423481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1314/2624] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:42.426320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:42.427871 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:42.443368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83c625-c436-4acc-b3fe-15d31deee13e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:42.448791 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:42.449466 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:42.451427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:42.451782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:42.452565 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:42.454483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:42.499827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:42.505891 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260b96e94c394d8b871cc592da906fbf', 'request_id': 'req-187a92c9-16b8-4fd4-a039-19115175e950'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.511968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260b96e94c394d8b871cc592da906fbf', 'request_id': 'req-187a92c9-16b8-4fd4-a039-19115175e950'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.517985 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260b96e94c394d8b871cc592da906fbf', 'request_id': 'req-187a92c9-16b8-4fd4-a039-19115175e950'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.575970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1315/2625] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:42 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:43:42.580006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:42.607732 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] The token cache is being invalidate because project 260b96e94c394d8b871cc592da906fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:42.608023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.608642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.610232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1311/2626] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:42 2022] DELETE /identity/v3/projects/260b96e94c394d8b871cc592da906fbf => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:42.615801 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:42.617319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:42.641502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492dc0a-9397-4371-b5ef-08932b2af873 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:42.646227 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:42.647041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:42.647366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:42.647648 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:42.649091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:42.651046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:42.681683 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a92c9-16b8-4fd4-a039-19115175e950 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:42.686462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:42.687400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:42.687689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:42.688008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] PATH_INFO: `/v3/users/aeca0908494c408b821ba3ecada46cbb` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:42.689195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeca0908494c408b821ba3ecada46cbb)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:42.691135 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:42.757250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1316/2627] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:42 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:43:42.761667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:42.763289 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:42.775524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc026-702d-4989-8066-8404e5de5193 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:42.780519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:42.781047 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:42.781335 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:42.781615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:42.782342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:42.783979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:42.799041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1317/2628] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:43:42.803368 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:42.814120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13414c-5d0f-4b4f-9d9b-a42af8fba9d0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:42.818831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:42.820611 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:42.821090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:42.821382 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] PATH_INFO: `/v3/projects/78778587a54b41a9bb5c6929de385dc2/users/e96f77f4d7b84d83a76fd405aa9d8e29/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:42.837250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78778587a54b41a9bb5c6929de385dc2, user_id=e96f77f4d7b84d83a76fd405aa9d8e29, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:42.840209 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:42.895974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeca0908494c408b821ba3ecada46cbb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.896621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeca0908494c408b821ba3ecada46cbb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.903548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeca0908494c408b821ba3ecada46cbb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.904226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeca0908494c408b821ba3ecada46cbb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:42.934482 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1318/2629] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:42 2022] PUT /identity/v3/projects/78778587a54b41a9bb5c6929de385dc2/users/e96f77f4d7b84d83a76fd405aa9d8e29/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:42.938388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:42.940050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:42.963146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:43.003501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d87b5-4470-4362-964d-c201942964e9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:43.008737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:43.009396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.009687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.009961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.010687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:43.015417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:43.021864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1319/2630] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:43:43.026055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:43.035627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6e31e-3211-4a3c-abe9-22c2b117b07d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:43.036278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeca0908494c408b821ba3ecada46cbb'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:43.040501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380f4f3-19e9-4fbb-a032-7f15a6d48d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:43.041539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380f4f3-19e9-4fbb-a032-7f15a6d48d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.045839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380f4f3-19e9-4fbb-a032-7f15a6d48d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.045839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380f4f3-19e9-4fbb-a032-7f15a6d48d5e admin admin] PATH_INFO: `/v3/projects/78778587a54b41a9bb5c6929de385dc2/users/e96f77f4d7b84d83a76fd405aa9d8e29/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.045839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380f4f3-19e9-4fbb-a032-7f15a6d48d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78778587a54b41a9bb5c6929de385dc2, user_id=e96f77f4d7b84d83a76fd405aa9d8e29, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:43.046905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380f4f3-19e9-4fbb-a032-7f15a6d48d5e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:43.055343 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1312/2631] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:42 2022] DELETE /identity/v3/users/aeca0908494c408b821ba3ecada46cbb => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:43.057561 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1320/2632] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:43 2022] PUT /identity/v3/projects/78778587a54b41a9bb5c6929de385dc2/users/e96f77f4d7b84d83a76fd405aa9d8e29/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:43.546264 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:43.547868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:43.629373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573e68-28e4-4270-99d7-9a8ed55c9984 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:43.634675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:43.635579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.635870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.636154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] PATH_INFO: `/v3/projects/bcce9424860140feaa2ca69a4a04e63d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.637478 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcce9424860140feaa2ca69a4a04e63d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:43.639440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:43.734867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcce9424860140feaa2ca69a4a04e63d', 'request_id': 'req-e61f8020-caf4-4328-a244-1836e39ff3ff'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:43.740718 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcce9424860140feaa2ca69a4a04e63d', 'request_id': 'req-e61f8020-caf4-4328-a244-1836e39ff3ff'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:43.747226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcce9424860140feaa2ca69a4a04e63d', 'request_id': 'req-e61f8020-caf4-4328-a244-1836e39ff3ff'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:43.807632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] The token cache is being invalidate because project bcce9424860140feaa2ca69a4a04e63d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:43.807929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:43.808571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e61f8020-caf4-4328-a244-1836e39ff3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:43.810268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1313/2633] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:43 2022] DELETE /identity/v3/projects/bcce9424860140feaa2ca69a4a04e63d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:43.816329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b4ff-a2b8-4277-893e-c6d1bd52c403 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.816818 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b4ff-a2b8-4277-893e-c6d1bd52c403 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.816907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b4ff-a2b8-4277-893e-c6d1bd52c403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.837211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-0668b4ff-a2b8-4277-893e-c6d1bd52c403 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:43.839739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0668b4ff-a2b8-4277-893e-c6d1bd52c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:43.900207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1321/2634] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:43.908444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3f980-7fad-47c7-bac6-9881949cdb0c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.908841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3f980-7fad-47c7-bac6-9881949cdb0c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.909192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3f980-7fad-47c7-bac6-9881949cdb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.928278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-8ff3f980-7fad-47c7-bac6-9881949cdb0c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:43.929761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8ff3f980-7fad-47c7-bac6-9881949cdb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:43.934662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:43.939416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1314/2635] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:43.943743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668b4ff-a2b8-4277-893e-c6d1bd52c403 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:43.954105 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668b4ff-a2b8-4277-893e-c6d1bd52c403 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:43.959066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:43.959858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.960160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.961162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.961836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:43.962180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:43.964114 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:43.979109 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1322/2636] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:43.984163 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3f980-7fad-47c7-bac6-9881949cdb0c None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:43.990466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3f980-7fad-47c7-bac6-9881949cdb0c None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:43.995350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd44a35-3355-4e90-ac9b-c0eae8a14c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:43.996020 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd44a35-3355-4e90-ac9b-c0eae8a14c65 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.996336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd44a35-3355-4e90-ac9b-c0eae8a14c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.996618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd44a35-3355-4e90-ac9b-c0eae8a14c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.997348 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd44a35-3355-4e90-ac9b-c0eae8a14c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:43.999338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd44a35-3355-4e90-ac9b-c0eae8a14c65 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.083140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1315/2637] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:43 2022] POST /identity/v3/projects => generated 391 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:43:44.088788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.098774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02db5a-4409-4e3f-b934-adf2ea6b3f00 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.160551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.161161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.161488 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.161773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.162484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.165140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.197338 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:44.283398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.283694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.283973 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.296156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1323/2638] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:44 2022] POST /identity/v3/users => generated 296 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:44.300267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.311401 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c2cf7-9092-40a5-99eb-e0e81720c095 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.313868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] MFA Rules not processed for user `e96f77f4d7b84d83a76fd405aa9d8e29`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:44.315152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:44.318939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.318939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.318939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.318939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.319280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.320795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.334575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1324/2639] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:44.336198 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.346548 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a1390-504c-4d06-a1ba-f96ff4b37cf7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.351440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.352221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.352510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.352790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.353520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.355132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.365832 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1325/2640] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:43:44.370433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.380023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710898cb-5d64-4623-8af0-34db35db55ec admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.385504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe976676-8f03-428e-af5d-059210bbf0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.386376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe976676-8f03-428e-af5d-059210bbf0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.386669 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe976676-8f03-428e-af5d-059210bbf0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.386951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe976676-8f03-428e-af5d-059210bbf0e4 admin admin] PATH_INFO: `/v3/projects/efc5c457c8434a7cb899f89c33498b87/users/e4ae316535e448f2b5cbd70697f03418/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.392337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1316/2641] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:44.401987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe976676-8f03-428e-af5d-059210bbf0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc5c457c8434a7cb899f89c33498b87, user_id=e4ae316535e448f2b5cbd70697f03418, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.404415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.404724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe976676-8f03-428e-af5d-059210bbf0e4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.405811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:44.462953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc75e-f58c-4217-b5e2-0074a42c9554 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.466796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.467476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.467766 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.473492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.473492 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:44.523201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1326/2642] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:44 2022] PUT /identity/v3/projects/efc5c457c8434a7cb899f89c33498b87/users/e4ae316535e448f2b5cbd70697f03418/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:44.526341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.526700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.526984 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.548449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.549968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.560403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1317/2643] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:43:44.565027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] MFA Rules not processed for user `e4ae316535e448f2b5cbd70697f03418`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:44.566064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:44.632783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1327/2644] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:44.637250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.648870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2dd64-b6d3-438d-8495-41709fa3742c service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.659197 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:44.708912 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.709605 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.709889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.710791 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.711649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.714860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.754668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.756099 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:44.834282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1318/2645] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:44 2022] POST /identity/v3/projects => generated 389 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:44.839541 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:44.841773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:44.851044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdec82f-900b-44e0-bb35-ad66a9734f0d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.904871 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a47cd-0f51-44bb-a3b0-98d1cb6359e8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.909909 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.910581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.910866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.911143 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.911893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.913796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.918000 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.918664 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.918952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.919864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.921051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:44.933326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.936241 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.946072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1328/2646] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:43:45.005271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1319/2647] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:44 2022] POST /identity/v3/users => generated 301 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:43:45.006729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:45.017784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179676-1662-4429-8df1-cbd891128d91 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:45.022462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:45.023082 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:45.023426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:45.023704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:45.024423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:45.026011 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:45.035792 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1329/2648] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:45.040145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:45.050233 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a0673-2116-4755-b84a-cfbb8e657c42 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:45.055593 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:45.056677 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:45.056971 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:45.057250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] PATH_INFO: `/v3/projects/ba007598fbe34aa68ad890281a92bcbe/users/b875d9c657274045bfa23c986f341022/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:45.072132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba007598fbe34aa68ad890281a92bcbe, user_id=b875d9c657274045bfa23c986f341022, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:45.074765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:45.156734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1320/2649] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:45 2022] PUT /identity/v3/projects/ba007598fbe34aa68ad890281a92bcbe/users/b875d9c657274045bfa23c986f341022/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:45.160881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:45.170989 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637a068-2a50-481a-9319-bc67b629bce8 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:45.202488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:45.227886 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9939d45-96ab-4842-a338-2c6f8ab3143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:45.228657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9939d45-96ab-4842-a338-2c6f8ab3143a admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:45.228948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9939d45-96ab-4842-a338-2c6f8ab3143a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:45.229355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9939d45-96ab-4842-a338-2c6f8ab3143a admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:45.229992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9939d45-96ab-4842-a338-2c6f8ab3143a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:45.231625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9939d45-96ab-4842-a338-2c6f8ab3143a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:45.242334 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1330/2650] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:45.246422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:45.257652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37440636-e6e8-47de-be58-57c326eea437 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:45.262388 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:45.263468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:45.263756 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:45.264037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] PATH_INFO: `/v3/projects/ba007598fbe34aa68ad890281a92bcbe/users/b875d9c657274045bfa23c986f341022/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:45.266012 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba007598fbe34aa68ad890281a92bcbe, user_id=b875d9c657274045bfa23c986f341022, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:45.268674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:45.280423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1321/2651] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:45 2022] PUT /identity/v3/projects/ba007598fbe34aa68ad890281a92bcbe/users/b875d9c657274045bfa23c986f341022/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:45.290909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:45.291233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:45.291518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:45.317755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] MFA Rules not processed for user `b875d9c657274045bfa23c986f341022`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:45.318952 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:45.388558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:45.392605 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1331/2652] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:45.403674 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:45.459701 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b84366-3728-41b4-9711-a08e302255f7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:45.463953 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:45.464628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:45.464910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:45.465187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:45.474458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:45.476638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:45.486692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1322/2653] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:45.540373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:45.541805 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:45.554725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5bddc-157c-4172-b59f-1f27e0a28ac9 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:45.558304 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:45.558948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:45.559270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:45.559551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:45.631671 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:45.633274 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:45.643568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1332/2654] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:46.458144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:46.467640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d74bf3-a4e9-4ce2-9f52-78daceb8d751 service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:46.471218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64877f-8917-445d-998c-a3c85b08e59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:46.476185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64877f-8917-445d-998c-a3c85b08e59b service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:46.476185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64877f-8917-445d-998c-a3c85b08e59b service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:46.476185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64877f-8917-445d-998c-a3c85b08e59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:46.539399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64877f-8917-445d-998c-a3c85b08e59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:46.540919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64877f-8917-445d-998c-a3c85b08e59b service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:46.550461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1323/2655] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:53.173065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:53.244876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164c4e2-bcd4-43e2-8eeb-4e92eebd4fd8 service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:53.247766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:53.248824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:53.249157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:53.249534 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] PATH_INFO: `/v3/users/e4ae316535e448f2b5cbd70697f03418` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:53.250870 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ae316535e448f2b5cbd70697f03418)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:53.254146 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:53.485162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ae316535e448f2b5cbd70697f03418'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:53.492182 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ae316535e448f2b5cbd70697f03418'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:53.492862 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ae316535e448f2b5cbd70697f03418'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:53.493473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ae316535e448f2b5cbd70697f03418'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:53.505738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ae316535e448f2b5cbd70697f03418'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:53.609837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1333/2656] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:53 2022] DELETE /identity/v3/users/e4ae316535e448f2b5cbd70697f03418 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:53.612406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:53.612894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:53.613115 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:53.631398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:53.632612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:53.692712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1324/2657] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:53.703073 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:53.706137 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:53.724264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:53.769323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06df90-2c4f-4116-8983-494c4f0ab9f9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:53.769323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:53.769323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:53.769693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:53.769866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:53.779001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:53.780518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:53.790091 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1334/2658] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:43:54.182542 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:54.184883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:54.193207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:54.201038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14703803-c6b7-4284-99ce-d2613f8073c4 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:54.207146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:54.207146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:54.207146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:54.207534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] PATH_INFO: `/v3/projects/efc5c457c8434a7cb899f89c33498b87` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:54.211476 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc5c457c8434a7cb899f89c33498b87)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:54.213970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:54.322373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc5c457c8434a7cb899f89c33498b87', 'request_id': 'req-0b6404c0-8097-4ba7-9739-f9b68a922f0a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.328074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc5c457c8434a7cb899f89c33498b87', 'request_id': 'req-0b6404c0-8097-4ba7-9739-f9b68a922f0a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.336520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc5c457c8434a7cb899f89c33498b87', 'request_id': 'req-0b6404c0-8097-4ba7-9739-f9b68a922f0a'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.436129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] The token cache is being invalidate because project efc5c457c8434a7cb899f89c33498b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:54.436423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.437079 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.438636 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1325/2659] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:54 2022] DELETE /identity/v3/projects/efc5c457c8434a7cb899f89c33498b87 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:54.442582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:54.452296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f7bf8-c3b1-4ab1-a47e-e59f1b2b6e86 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:54.508329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:54.509230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:54.510050 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:54.510344 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] PATH_INFO: `/v3/users/b875d9c657274045bfa23c986f341022` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:54.511577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b875d9c657274045bfa23c986f341022)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:54.513575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:54.712005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b875d9c657274045bfa23c986f341022'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.719588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b875d9c657274045bfa23c986f341022'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.720245 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b875d9c657274045bfa23c986f341022'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.720857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b875d9c657274045bfa23c986f341022'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.732677 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-0fa2ec09-956d-4d49-99fc-2054493aa0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b875d9c657274045bfa23c986f341022'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:54.846924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1335/2660] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:43:54 2022] DELETE /identity/v3/users/b875d9c657274045bfa23c986f341022 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:55.272009 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:55.273699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:55.287635 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6404c0-8097-4ba7-9739-f9b68a922f0a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:55.367785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:55.370795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.370795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.370795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] PATH_INFO: `/v3/projects/ba007598fbe34aa68ad890281a92bcbe` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.374632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba007598fbe34aa68ad890281a92bcbe)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:55.374632 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:55.609597 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba007598fbe34aa68ad890281a92bcbe', 'request_id': 'req-e1e54350-f726-4d3a-95b8-e0374cea2f9b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:55.615057 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba007598fbe34aa68ad890281a92bcbe', 'request_id': 'req-e1e54350-f726-4d3a-95b8-e0374cea2f9b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:55.621855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba007598fbe34aa68ad890281a92bcbe', 'request_id': 'req-e1e54350-f726-4d3a-95b8-e0374cea2f9b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:55.721232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] The token cache is being invalidate because project ba007598fbe34aa68ad890281a92bcbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:43:55.721628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:55.722277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e1e54350-f726-4d3a-95b8-e0374cea2f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:43:55.724292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1326/2661] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:43:55 2022] DELETE /identity/v3/projects/ba007598fbe34aa68ad890281a92bcbe => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:55.729943 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78323-a13a-40de-9542-570ec97db4d4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.730381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78323-a13a-40de-9542-570ec97db4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.730507 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78323-a13a-40de-9542-570ec97db4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.750100 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-31e78323-a13a-40de-9542-570ec97db4d4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:55.754895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-31e78323-a13a-40de-9542-570ec97db4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:55.785616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:55.815237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1336/2662] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:55.823095 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded905-b02c-424d-a186-be22dbd849ec None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.823447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded905-b02c-424d-a186-be22dbd849ec None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.823737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded905-b02c-424d-a186-be22dbd849ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.845003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-89ded905-b02c-424d-a186-be22dbd849ec None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:55.846354 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-89ded905-b02c-424d-a186-be22dbd849ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:55.872575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1327/2663] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:43:55.872784 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e78323-a13a-40de-9542-570ec97db4d4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:55.877322 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e78323-a13a-40de-9542-570ec97db4d4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:55.882822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:55.883854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.883854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.884163 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.884959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:55.885249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:55.886885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:55.898829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1337/2664] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:55.904722 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ded905-b02c-424d-a186-be22dbd849ec None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:55.914325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ded905-b02c-424d-a186-be22dbd849ec None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:55.920449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:55.920862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.921158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.921439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.922146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:55.924142 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:55.932856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:56.066280 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1328/2665] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:43:55 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:56.070080 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.080894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024c7cf-6ff6-44c3-970c-af909d6e3fb3 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.142660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.143331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.143619 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.143907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.144603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.146570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.220808 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1338/2666] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:43:56 2022] POST /identity/v3/users => generated 285 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:43:56.224959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.235974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7705e-b0a3-44ca-b834-a32b6cb71844 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.241075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.241795 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.249787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.249787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.249787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.249787 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.255047 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1329/2667] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:56.259839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.270233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96838ecf-e9d7-492f-a0e3-916685f72845 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.275607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8cd0a-08de-4a94-a618-b1753fb82c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.278556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8cd0a-08de-4a94-a618-b1753fb82c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.278556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8cd0a-08de-4a94-a618-b1753fb82c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.278556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8cd0a-08de-4a94-a618-b1753fb82c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.278556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8cd0a-08de-4a94-a618-b1753fb82c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.279172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8cd0a-08de-4a94-a618-b1753fb82c98 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.289165 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1339/2668] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:43:56.293544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.303499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1310945-0d54-459e-8f6a-14751a2c17ef admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.308125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.309042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.309332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.309614 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] PATH_INFO: `/v3/projects/f7e5e21696c14bf4b10133c7669b146c/users/a3e051767bf9433ba1e856042be4cf19/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.324706 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e5e21696c14bf4b10133c7669b146c, user_id=a3e051767bf9433ba1e856042be4cf19, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.327444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.448698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1330/2669] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:43:56 2022] PUT /identity/v3/projects/f7e5e21696c14bf4b10133c7669b146c/users/a3e051767bf9433ba1e856042be4cf19/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:56.463115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.463465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.463747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.490524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] MFA Rules not processed for user `a3e051767bf9433ba1e856042be4cf19`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:56.491779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:56.557773 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1340/2670] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:43:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:43:56.569324 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.571070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:56.632166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640319ef-25a7-4234-8513-2d29cc0d8c2a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.637074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '282c8662f2a2431b97d1bf122eabf5a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.637725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.638018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.638304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.646979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.648466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.658136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1331/2671] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:43:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:03.859779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:03.861319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:03.904616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:03.940572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ff944-ea09-4f43-b362-5b7c3d60080f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:03.945962 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:03.946578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:03.946868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:03.947150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:03.949046 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:03.963040 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:03.965013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:03.978655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1341/2672] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:04.224101 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:04.229802 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:04.287796 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405575c-5e88-484b-b6cc-c5913f7ea2ae service glance] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:04.292002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:04.292660 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:04.293466 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:04.293842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:04.301334 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:04.370278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:04.371811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:04.381975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1332/2673] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:04.667345 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:04.669759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:04.690325 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0404-00d6-4c30-b9e5-5903cc044b2e service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:04.693595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:04.694490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:04.694783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:04.695069 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] PATH_INFO: `/v3/users/cb6e4992e2d04aee8eed8f2c7314568c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:04.696354 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb6e4992e2d04aee8eed8f2c7314568c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:04.701542 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:04.849187 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb6e4992e2d04aee8eed8f2c7314568c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:04.857252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb6e4992e2d04aee8eed8f2c7314568c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:04.857950 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb6e4992e2d04aee8eed8f2c7314568c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:04.858552 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb6e4992e2d04aee8eed8f2c7314568c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:04.873672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb6e4992e2d04aee8eed8f2c7314568c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:04.961027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1342/2674] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:04 2022] DELETE /identity/v3/users/cb6e4992e2d04aee8eed8f2c7314568c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:05.065594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.067082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:05.079998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209fc1-edf4-4956-996e-7561694e8833 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:05.084244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:05.160029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:05.160920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.161209 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.161491 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] PATH_INFO: `/v3/users/8e3d8bedb0804162b5f20956022d4083` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.175854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3d8bedb0804162b5f20956022d4083)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:05.177828 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:05.225670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.227077 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:05.241468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb868122-341c-462e-a615-264aab381a8f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:05.246575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:05.247583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.247869 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.248143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] PATH_INFO: `/v3/projects/5e01c4c716ee4d128955d5a21b328155` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.249587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e01c4c716ee4d128955d5a21b328155)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:05.251663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:05.387407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3d8bedb0804162b5f20956022d4083'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.387959 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e01c4c716ee4d128955d5a21b328155', 'request_id': 'req-bc8c868b-d0d3-4772-8e6d-601f924c7585'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.388089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3d8bedb0804162b5f20956022d4083'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.393544 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e01c4c716ee4d128955d5a21b328155', 'request_id': 'req-bc8c868b-d0d3-4772-8e6d-601f924c7585'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.395100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3d8bedb0804162b5f20956022d4083'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.395788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3d8bedb0804162b5f20956022d4083'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.400364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e01c4c716ee4d128955d5a21b328155', 'request_id': 'req-bc8c868b-d0d3-4772-8e6d-601f924c7585'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.475679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3d8bedb0804162b5f20956022d4083'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.494974 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1333/2675] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:05 2022] DELETE /identity/v3/users/8e3d8bedb0804162b5f20956022d4083 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:05.520484 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] The token cache is being invalidate because project 5e01c4c716ee4d128955d5a21b328155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:05.520903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.521649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.523279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1343/2676] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:05 2022] DELETE /identity/v3/projects/5e01c4c716ee4d128955d5a21b328155 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:05.528050 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.529570 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:05.604633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b866b-7ce0-4daa-aa27-8cd0b43891a8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:05.610120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:05.611015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.611341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.611627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] PATH_INFO: `/v3/users/e96f77f4d7b84d83a76fd405aa9d8e29` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.612876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e96f77f4d7b84d83a76fd405aa9d8e29)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:05.614835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:05.632390 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:05.729961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:05.731875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:05.761066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:05.798680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e96f77f4d7b84d83a76fd405aa9d8e29'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.799444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e96f77f4d7b84d83a76fd405aa9d8e29'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.808365 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e96f77f4d7b84d83a76fd405aa9d8e29'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.817997 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e96f77f4d7b84d83a76fd405aa9d8e29'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.818908 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c868b-d0d3-4772-8e6d-601f924c7585 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:05.823743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:05.825313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:05.825796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:05.825995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] PATH_INFO: `/v3/projects/d63d0dda45964a9badc7d1897ead09e7` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:05.838822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63d0dda45964a9badc7d1897ead09e7)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:05.841473 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:05.881839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e96f77f4d7b84d83a76fd405aa9d8e29'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.906449 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1334/2677] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:05 2022] DELETE /identity/v3/users/e96f77f4d7b84d83a76fd405aa9d8e29 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:05.982783 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63d0dda45964a9badc7d1897ead09e7', 'request_id': 'req-30a1f7ca-d536-4184-8094-41d39e672c6c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.989311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63d0dda45964a9badc7d1897ead09e7', 'request_id': 'req-30a1f7ca-d536-4184-8094-41d39e672c6c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:05.998763 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63d0dda45964a9badc7d1897ead09e7', 'request_id': 'req-30a1f7ca-d536-4184-8094-41d39e672c6c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.087848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] The token cache is being invalidate because project d63d0dda45964a9badc7d1897ead09e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:06.088205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.088852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-30a1f7ca-d536-4184-8094-41d39e672c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.090460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1344/2678] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:05 2022] DELETE /identity/v3/projects/d63d0dda45964a9badc7d1897ead09e7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:06.180535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:06.182484 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:06.284375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303b61d-cb38-4b5e-9bb6-94f3fef05425 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:06.341905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:06.342703 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.342992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.343311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] PATH_INFO: `/v3/projects/78778587a54b41a9bb5c6929de385dc2` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.344499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] RBAC: Authorizing `identity:delete_project(project_id=78778587a54b41a9bb5c6929de385dc2)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:06.346399 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:06.510304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:06.514561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78778587a54b41a9bb5c6929de385dc2', 'request_id': 'req-2177abd6-2643-4494-8808-e2bb180f726d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.520032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78778587a54b41a9bb5c6929de385dc2', 'request_id': 'req-2177abd6-2643-4494-8808-e2bb180f726d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.526288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78778587a54b41a9bb5c6929de385dc2', 'request_id': 'req-2177abd6-2643-4494-8808-e2bb180f726d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.617785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] The token cache is being invalidate because project 78778587a54b41a9bb5c6929de385dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:06.618090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.618712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2177abd6-2643-4494-8808-e2bb180f726d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:06.620521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1335/2679] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:06 2022] DELETE /identity/v3/projects/78778587a54b41a9bb5c6929de385dc2 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:06.625587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445af388-4cfb-4707-a166-b3c7b6b3bc25 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.625917 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445af388-4cfb-4707-a166-b3c7b6b3bc25 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.626306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445af388-4cfb-4707-a166-b3c7b6b3bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.647120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-445af388-4cfb-4707-a166-b3c7b6b3bc25 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:06.651864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-445af388-4cfb-4707-a166-b3c7b6b3bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:06.716339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1345/2680] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:06.724231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd18b4-23c6-4efb-b7cb-7e8d6d638aac None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.724524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd18b4-23c6-4efb-b7cb-7e8d6d638aac None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.724808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd18b4-23c6-4efb-b7cb-7e8d6d638aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.745087 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c4cd18b4-23c6-4efb-b7cb-7e8d6d638aac None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:06.746473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c4cd18b4-23c6-4efb-b7cb-7e8d6d638aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:06.757968 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1336/2681] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:06.762512 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445af388-4cfb-4707-a166-b3c7b6b3bc25 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:06.772444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445af388-4cfb-4707-a166-b3c7b6b3bc25 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:06.777030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:06.778223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.778506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.778799 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.779609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:06.779933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:06.781519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:06.793891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1346/2682] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:06.798299 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd18b4-23c6-4efb-b7cb-7e8d6d638aac None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:06.809576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd18b4-23c6-4efb-b7cb-7e8d6d638aac None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:06.814332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:06.815023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:06.815345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:06.815627 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:06.816344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:06.818242 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:06.945540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1337/2683] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:06 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:06.952975 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:06.960033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae057ce-b0e6-4721-9140-59c5f6f8aaef admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:06.988115 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:07.019617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.020287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.020576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.020858 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.021551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.023683 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.115873 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1347/2684] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:06 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:07.119998 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.130693 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebd7b9-4f42-43cf-86fb-94526758c28a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.135759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.136427 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.137378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.137854 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.138712 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.140637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.151737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1338/2685] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:07.156386 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.166615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac1b1f-0eed-4bb2-a666-97501fc769a6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.171566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c300025-72ba-44c5-8e9a-8590a983017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.172199 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300025-72ba-44c5-8e9a-8590a983017e admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.172483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300025-72ba-44c5-8e9a-8590a983017e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.172764 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c300025-72ba-44c5-8e9a-8590a983017e admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.173517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c300025-72ba-44c5-8e9a-8590a983017e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.175119 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c300025-72ba-44c5-8e9a-8590a983017e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.185904 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1348/2686] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:07.190137 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.201383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe210f-ed7c-4e7c-aa9f-a765ee713318 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.206129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.207024 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.207350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.207634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] PATH_INFO: `/v3/projects/c4cb48202a13413bbea53cb99c11f7e8/users/fe70389332a348beb04aaf12864342e5/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.224078 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cb48202a13413bbea53cb99c11f7e8, user_id=fe70389332a348beb04aaf12864342e5, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.227671 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.328028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1339/2687] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:07 2022] PUT /identity/v3/projects/c4cb48202a13413bbea53cb99c11f7e8/users/fe70389332a348beb04aaf12864342e5/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:07.341312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48eeb4-2316-4521-a2f0-b114aa99aa98 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.341610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48eeb4-2316-4521-a2f0-b114aa99aa98 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.341894 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48eeb4-2316-4521-a2f0-b114aa99aa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.368250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1f48eeb4-2316-4521-a2f0-b114aa99aa98 None None] MFA Rules not processed for user `fe70389332a348beb04aaf12864342e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:07.369449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1f48eeb4-2316-4521-a2f0-b114aa99aa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:07.435101 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1349/2688] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:07.439452 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.504192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535e8d-214f-4b04-a083-ae3471e073bb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.509533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.510431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.510991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.511234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.512738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.514694 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.592282 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1340/2689] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:07 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:07.596463 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48eeb4-2316-4521-a2f0-b114aa99aa98 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.608298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48eeb4-2316-4521-a2f0-b114aa99aa98 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.668171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.668947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.669243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.669606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.670224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.672219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.783743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1350/2690] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:07 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:07.787626 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.801292 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbea08-f67c-4537-a91d-10a3a4b3ac3b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.806049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.806714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.807003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.807329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.808063 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.811002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.821534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1341/2691] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:07.825514 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.835606 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18ccec-d6c3-4574-8a2b-d0fbac86aff6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.840665 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cbb85-57bc-4828-be23-dbdfe9b943e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.841320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cbb85-57bc-4828-be23-dbdfe9b943e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.841603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cbb85-57bc-4828-be23-dbdfe9b943e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.841883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cbb85-57bc-4828-be23-dbdfe9b943e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.842633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cbb85-57bc-4828-be23-dbdfe9b943e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.844248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cbb85-57bc-4828-be23-dbdfe9b943e9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.854062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1351/2692] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:07.858897 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:07.869008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958edb9-ab0a-479a-8f84-8a422d3b69dc admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:07.874232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:07.875108 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.875421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.875702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] PATH_INFO: `/v3/projects/6a02934426134db585a617efe805a54e/users/b8493b75d12144ecbcf3455827bc3f57/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:07.890145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a02934426134db585a617efe805a54e, user_id=b8493b75d12144ecbcf3455827bc3f57, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:07.892907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:07.967740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1342/2693] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:07 2022] PUT /identity/v3/projects/6a02934426134db585a617efe805a54e/users/b8493b75d12144ecbcf3455827bc3f57/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:07.978419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b653992-2d4e-4e7c-82ee-36cf328cbbeb None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:07.978715 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b653992-2d4e-4e7c-82ee-36cf328cbbeb None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:07.978999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b653992-2d4e-4e7c-82ee-36cf328cbbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.006460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7b653992-2d4e-4e7c-82ee-36cf328cbbeb None None] MFA Rules not processed for user `b8493b75d12144ecbcf3455827bc3f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:08.007705 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7b653992-2d4e-4e7c-82ee-36cf328cbbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:08.011766 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:08.072358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1352/2694] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:08.076814 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.097566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:08.144392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7856e-630c-4060-a93b-9b8bb97cbe39 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.149316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.149988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.150278 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.150561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.151344 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.154044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.226702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1343/2695] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:08 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:08.231539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b653992-2d4e-4e7c-82ee-36cf328cbbeb None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.242571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b653992-2d4e-4e7c-82ee-36cf328cbbeb None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.304435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.305111 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.305396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.305676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.306365 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.308310 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.390084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1353/2696] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:08 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:08.394252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.405223 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac7c76-9add-4ad6-a716-ab0065f13150 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.409984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.410641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.410982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.411398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.412346 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.414561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.427607 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1344/2697] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:08.432651 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.443910 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388b07b-1ac8-4fdf-be3d-b8f252ad46bd admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.448574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.449462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.449748 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.450024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] PATH_INFO: `/v3/projects/05895d625f72459d8a35e9c262d3600a/users/19d0df094c554a55abadfcf1745d2dd4/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.466432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05895d625f72459d8a35e9c262d3600a, user_id=19d0df094c554a55abadfcf1745d2dd4, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.469116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.572604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1354/2698] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:08 2022] PUT /identity/v3/projects/05895d625f72459d8a35e9c262d3600a/users/19d0df094c554a55abadfcf1745d2dd4/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:08.577288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.588228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa9b45-b2ec-45fc-9ddb-881db0e007f8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.645040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a62f26-4649-4cae-840f-6cc16478a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.645631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a62f26-4649-4cae-840f-6cc16478a044 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.645920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a62f26-4649-4cae-840f-6cc16478a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.646202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a62f26-4649-4cae-840f-6cc16478a044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.646924 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a62f26-4649-4cae-840f-6cc16478a044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.648713 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a62f26-4649-4cae-840f-6cc16478a044 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.659104 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1345/2699] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:08.663208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.673866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a44c1-cdcc-4094-9796-4eef8c13d1b7 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.678468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.679384 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.679667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.679941 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] PATH_INFO: `/v3/projects/05895d625f72459d8a35e9c262d3600a/users/19d0df094c554a55abadfcf1745d2dd4/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.681879 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] RBAC: Authorizing `identity:create_grant(project_id=05895d625f72459d8a35e9c262d3600a, user_id=19d0df094c554a55abadfcf1745d2dd4, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.684765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.695378 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1355/2700] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:08 2022] PUT /identity/v3/projects/05895d625f72459d8a35e9c262d3600a/users/19d0df094c554a55abadfcf1745d2dd4/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:08.705865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dc46c-0f7d-47ff-816c-b1b774bec355 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.706166 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dc46c-0f7d-47ff-816c-b1b774bec355 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.706455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738dc46c-0f7d-47ff-816c-b1b774bec355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.733696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-738dc46c-0f7d-47ff-816c-b1b774bec355 None None] MFA Rules not processed for user `19d0df094c554a55abadfcf1745d2dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:08.734934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-738dc46c-0f7d-47ff-816c-b1b774bec355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:08.801967 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1346/2701] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:08.813342 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:08.815041 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:08.872394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e5794-3a45-472a-a92a-a7ce2576114e admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:08.875997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:08.876630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:08.876913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:08.877189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:08.886294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:08.933935 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:08.935603 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:08.944429 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1356/2702] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:12.817467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738dc46c-0f7d-47ff-816c-b1b774bec355 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:12.828630 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738dc46c-0f7d-47ff-816c-b1b774bec355 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:12.832539 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:12.833193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:12.833525 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:12.833810 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:12.843913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:12.891422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:12.892908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:12.901736 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1347/2703] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:15.115202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:15.124809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709fff5-339f-46ed-9d2c-81dea89e798a service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:15.128510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d745306-1e2f-4271-9e23-3807cf7cfe0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:15.129156 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d745306-1e2f-4271-9e23-3807cf7cfe0f service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:15.129477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d745306-1e2f-4271-9e23-3807cf7cfe0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:15.129755 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d745306-1e2f-4271-9e23-3807cf7cfe0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:15.140181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d745306-1e2f-4271-9e23-3807cf7cfe0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:15.141620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d745306-1e2f-4271-9e23-3807cf7cfe0f service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:15.151474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1357/2704] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:19.602211 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:19.665615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb7a9e-dffd-4c45-bf3e-5ff772120fdf service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:19.670125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:19.671106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.672369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.672659 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] PATH_INFO: `/v3/users/fe70389332a348beb04aaf12864342e5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.673856 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe70389332a348beb04aaf12864342e5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:19.675823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:19.838682 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe70389332a348beb04aaf12864342e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.839442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe70389332a348beb04aaf12864342e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.847433 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe70389332a348beb04aaf12864342e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.848114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe70389332a348beb04aaf12864342e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.903708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe70389332a348beb04aaf12864342e5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:19.923865 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1348/2705] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:19 2022] DELETE /identity/v3/users/fe70389332a348beb04aaf12864342e5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:19.928547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:19.928991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:19.929132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:19.945796 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:19.946979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.006955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1358/2706] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:20.017518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.019093 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.053117 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:20.076587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad947c-af4a-4dc8-b80a-798d6142fd28 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.082221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.082771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.083088 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.083537 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.095279 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.096888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:20.109932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1349/2707] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:20.527916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.529377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:20.543641 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481825c7-93eb-4af9-add1-13a3484cdc1e None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.549076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.550094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.550390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.550676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] PATH_INFO: `/v3/projects/c4cb48202a13413bbea53cb99c11f7e8` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.551889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4cb48202a13413bbea53cb99c11f7e8)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.554224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:20.663874 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4cb48202a13413bbea53cb99c11f7e8', 'request_id': 'req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:20.669550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4cb48202a13413bbea53cb99c11f7e8', 'request_id': 'req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:20.677061 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4cb48202a13413bbea53cb99c11f7e8', 'request_id': 'req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:20.778519 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] The token cache is being invalidate because project c4cb48202a13413bbea53cb99c11f7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:20.778809 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:20.779480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:20.781610 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1359/2708] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:20 2022] DELETE /identity/v3/projects/c4cb48202a13413bbea53cb99c11f7e8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:20.785540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:20.795351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfd4cd-5603-4d54-9bd1-ffa0b5556f90 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:20.852187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:20.852982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:20.853271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:20.853552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] PATH_INFO: `/v3/users/b8493b75d12144ecbcf3455827bc3f57` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:20.854733 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8493b75d12144ecbcf3455827bc3f57)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:20.856730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:21.169246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8493b75d12144ecbcf3455827bc3f57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.169903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8493b75d12144ecbcf3455827bc3f57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.177170 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8493b75d12144ecbcf3455827bc3f57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.177812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8493b75d12144ecbcf3455827bc3f57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.263312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8493b75d12144ecbcf3455827bc3f57'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.283205 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1350/2709] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:20 2022] DELETE /identity/v3/users/b8493b75d12144ecbcf3455827bc3f57 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:21.695125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:21.695738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:21.710436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c222f0b-7f56-40c5-810a-bad2a7cb9af0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:21.731341 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:21.774268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:21.775142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:21.775495 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:21.775775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] PATH_INFO: `/v3/projects/6a02934426134db585a617efe805a54e` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:21.777219 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a02934426134db585a617efe805a54e)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:21.779948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:21.881034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a02934426134db585a617efe805a54e', 'request_id': 'req-394ac6f0-8ceb-4bef-88ad-43f141430448'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.887361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a02934426134db585a617efe805a54e', 'request_id': 'req-394ac6f0-8ceb-4bef-88ad-43f141430448'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.895275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a02934426134db585a617efe805a54e', 'request_id': 'req-394ac6f0-8ceb-4bef-88ad-43f141430448'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.960600 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] The token cache is being invalidate because project 6a02934426134db585a617efe805a54e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:21.960948 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.961842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:21.963420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1360/2710] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:21 2022] DELETE /identity/v3/projects/6a02934426134db585a617efe805a54e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:21.967406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:21.968988 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:21.981758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a19dd5-d344-4bc1-92a0-434b87b7a2b8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:22.032558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:22.041006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:22.041885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:22.042173 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:22.042454 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] PATH_INFO: `/v3/users/19d0df094c554a55abadfcf1745d2dd4` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:22.043687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d0df094c554a55abadfcf1745d2dd4)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:22.046171 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:22.214383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d0df094c554a55abadfcf1745d2dd4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:22.215275 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d0df094c554a55abadfcf1745d2dd4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:22.222262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d0df094c554a55abadfcf1745d2dd4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:22.222979 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d0df094c554a55abadfcf1745d2dd4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:22.320631 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-70ce23bc-3a98-4d1a-a958-e1132817f941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d0df094c554a55abadfcf1745d2dd4'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:22.343410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1351/2711] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:21 2022] DELETE /identity/v3/users/19d0df094c554a55abadfcf1745d2dd4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:22.962422 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:22.963909 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:22.980028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ac6f0-8ceb-4bef-88ad-43f141430448 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.045104 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.046035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.046562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.047023 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] PATH_INFO: `/v3/projects/05895d625f72459d8a35e9c262d3600a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.048006 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05895d625f72459d8a35e9c262d3600a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.057234 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.178347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05895d625f72459d8a35e9c262d3600a', 'request_id': 'req-e848adf2-91ea-45db-ac76-5dabd729300c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.182238 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:23.185317 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05895d625f72459d8a35e9c262d3600a', 'request_id': 'req-e848adf2-91ea-45db-ac76-5dabd729300c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.192442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05895d625f72459d8a35e9c262d3600a', 'request_id': 'req-e848adf2-91ea-45db-ac76-5dabd729300c'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.350319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] The token cache is being invalidate because project 05895d625f72459d8a35e9c262d3600a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:23.350620 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.351290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e848adf2-91ea-45db-ac76-5dabd729300c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:23.352824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1361/2712] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:22 2022] DELETE /identity/v3/projects/05895d625f72459d8a35e9c262d3600a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:23.359239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a9b1-b7b9-4d94-9e30-d2902d7d0255 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.360332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a9b1-b7b9-4d94-9e30-d2902d7d0255 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.360544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a9b1-b7b9-4d94-9e30-d2902d7d0255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.384687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-1888a9b1-b7b9-4d94-9e30-d2902d7d0255 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:23.384687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1888a9b1-b7b9-4d94-9e30-d2902d7d0255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:23.446620 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1352/2713] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:23.454397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51690953-6b46-463b-b2b6-55b4fa3a669b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.457257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51690953-6b46-463b-b2b6-55b4fa3a669b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.457257 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51690953-6b46-463b-b2b6-55b4fa3a669b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.474441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-51690953-6b46-463b-b2b6-55b4fa3a669b None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:23.475684 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-51690953-6b46-463b-b2b6-55b4fa3a669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:23.485228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1362/2714] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:23.489544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888a9b1-b7b9-4d94-9e30-d2902d7d0255 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.499301 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888a9b1-b7b9-4d94-9e30-d2902d7d0255 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.504790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.505457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.505734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.506016 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.506741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:23.507134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.508768 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.520670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1353/2715] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:23.524878 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51690953-6b46-463b-b2b6-55b4fa3a669b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.534541 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51690953-6b46-463b-b2b6-55b4fa3a669b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.539543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.543154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.543154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.543154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.543154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.543506 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.619233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1363/2716] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:23 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:23.623048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.634160 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc6b0e-36f5-45db-981a-6acfbe61d8dd admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.693875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:23.727215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.727215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.727215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.727215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.727586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.727935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.811708 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1354/2717] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:23 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:23.815911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.849141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a9fd7-89dc-47c5-9cbd-0a1169885ca9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.854239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.854927 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.855252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.855533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.856323 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.857914 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.871856 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1364/2718] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:23.873041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.883347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f46eb-c619-43e7-b268-663ac5472353 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.888376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137ede8-3b0d-4646-8e8c-7d4ce366052a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.889023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137ede8-3b0d-4646-8e8c-7d4ce366052a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.889309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137ede8-3b0d-4646-8e8c-7d4ce366052a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.889588 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137ede8-3b0d-4646-8e8c-7d4ce366052a admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.890362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137ede8-3b0d-4646-8e8c-7d4ce366052a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.892023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137ede8-3b0d-4646-8e8c-7d4ce366052a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:23.904917 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1355/2719] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:23.906273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:23.917247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec020bf8-acec-4453-a568-d3968d91d33d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:23.922035 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:23.923578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:23.923865 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:23.924142 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] PATH_INFO: `/v3/projects/a9753aef45e649ac9fa4a0736ab43b7c/users/55b4ee7185a2437aaf5e29ff2cfce622/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:23.939130 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9753aef45e649ac9fa4a0736ab43b7c, user_id=55b4ee7185a2437aaf5e29ff2cfce622, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:23.941857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:24.029352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1365/2720] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:23 2022] PUT /identity/v3/projects/a9753aef45e649ac9fa4a0736ab43b7c/users/55b4ee7185a2437aaf5e29ff2cfce622/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:24.040445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:24.040741 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:24.041023 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:24.068153 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] MFA Rules not processed for user `55b4ee7185a2437aaf5e29ff2cfce622`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:24.069353 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:24.135475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1356/2721] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:24.147293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:24.149044 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:24.227007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec520-e7ca-4bad-bd47-c28ef60a74e4 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:24.230737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2915575-5074-4ad6-afe4-5f14474a70ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:24.231447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2915575-5074-4ad6-afe4-5f14474a70ad service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:24.231735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2915575-5074-4ad6-afe4-5f14474a70ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:24.232018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2915575-5074-4ad6-afe4-5f14474a70ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:24.241481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2915575-5074-4ad6-afe4-5f14474a70ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:24.242919 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2915575-5074-4ad6-afe4-5f14474a70ad service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:24.252971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1366/2722] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:33.838983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:33.840440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:33.915105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d77c2f-20cc-4601-97f4-2fa03978a096 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:33.919676 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:33.920548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:33.920837 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:33.921215 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] PATH_INFO: `/v3/users/8b80f488fb5d46f99df9845823280224` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:33.922407 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b80f488fb5d46f99df9845823280224)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:33.924367 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:34.090622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:34.098750 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b80f488fb5d46f99df9845823280224'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:34.099445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b80f488fb5d46f99df9845823280224'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:34.106168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b80f488fb5d46f99df9845823280224'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:34.111447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b80f488fb5d46f99df9845823280224'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:34.182193 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b80f488fb5d46f99df9845823280224'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:34.199960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1357/2723] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:33 2022] DELETE /identity/v3/users/8b80f488fb5d46f99df9845823280224 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:34.204662 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:34.205120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:34.205329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:34.223899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:34.225140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:34.269441 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:34.287546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1367/2724] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:34.297519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:34.300381 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:34.357411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87d8da-1059-420f-a95f-9d5bb137dd92 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:34.361075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:34.361717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:34.362003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:34.362900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:34.371999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:34.373415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:34.383403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1358/2725] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:34.747214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:34.748643 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:34.761590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeac84-881b-4afd-afeb-27c0f50c9f52 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:34.766220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:34.767088 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:34.767415 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:34.767694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] PATH_INFO: `/v3/users/a3e051767bf9433ba1e856042be4cf19` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:34.769008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e051767bf9433ba1e856042be4cf19)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:34.770925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:34.836244 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:34.837699 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:34.850962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eab86b-058b-4bbf-80b7-13c97a34fed3 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:34.855710 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:34.856610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:34.856896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:34.857180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] PATH_INFO: `/v3/projects/de76c80ac72c46c5a17755d6ac45a199` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:34.858374 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de76c80ac72c46c5a17755d6ac45a199)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:34.860451 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:35.038577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e051767bf9433ba1e856042be4cf19'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.038783 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de76c80ac72c46c5a17755d6ac45a199', 'request_id': 'req-bd7e38f8-6eb4-4927-9730-d78b75caee85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.044940 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de76c80ac72c46c5a17755d6ac45a199', 'request_id': 'req-bd7e38f8-6eb4-4927-9730-d78b75caee85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.045756 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e051767bf9433ba1e856042be4cf19'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.046428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e051767bf9433ba1e856042be4cf19'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.047065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e051767bf9433ba1e856042be4cf19'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.052083 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de76c80ac72c46c5a17755d6ac45a199', 'request_id': 'req-bd7e38f8-6eb4-4927-9730-d78b75caee85'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.060968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6f7308ca-622f-4e1f-bd30-b0782f58af92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e051767bf9433ba1e856042be4cf19'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.114601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] The token cache is being invalidate because project de76c80ac72c46c5a17755d6ac45a199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:35.114901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.115587 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.117152 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1359/2726] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:34 2022] DELETE /identity/v3/projects/de76c80ac72c46c5a17755d6ac45a199 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:35.121132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:35.122543 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:35.166094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1368/2727] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:34 2022] DELETE /identity/v3/users/a3e051767bf9433ba1e856042be4cf19 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:35.169230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:35.170618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:35.170913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:35.171270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:35.189221 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:35.191658 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:35.192228 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e38f8-6eb4-4927-9730-d78b75caee85 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:35.249681 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:35.253561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:35.253561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:35.253561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] PATH_INFO: `/v3/users/166e010c481e49a1b202704eafe2521c` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:35.253561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166e010c481e49a1b202704eafe2521c)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:35.254821 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:35.256595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1369/2728] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:35.266262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:35.267703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:35.326594 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19cfd-edca-4581-87e9-32517d91c879 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:35.330248 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:35.331412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:35.331709 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:35.331976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:35.341573 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:35.343019 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:35.353411 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1370/2729] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:35.454414 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166e010c481e49a1b202704eafe2521c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.455091 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166e010c481e49a1b202704eafe2521c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.463094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166e010c481e49a1b202704eafe2521c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.463771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166e010c481e49a1b202704eafe2521c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.547836 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166e010c481e49a1b202704eafe2521c'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:35.576033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1360/2730] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:35 2022] DELETE /identity/v3/users/166e010c481e49a1b202704eafe2521c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:35.785717 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:35.787369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:35.800670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:35.810356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22fff0-314f-438e-afb9-b0fc9ab5dba5 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:35.875460 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:35.876978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:35.877510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:35.878520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] PATH_INFO: `/v3/projects/f7e5e21696c14bf4b10133c7669b146c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:35.879780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e5e21696c14bf4b10133c7669b146c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:35.881712 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.024562 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.026469 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.039916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e5e21696c14bf4b10133c7669b146c', 'request_id': 'req-3d45c768-00f9-4340-86bb-1b4fdf460009'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.041144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823fa9-6729-478c-9eb5-5bfbf7937589 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.046419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e5e21696c14bf4b10133c7669b146c', 'request_id': 'req-3d45c768-00f9-4340-86bb-1b4fdf460009'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.053253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e5e21696c14bf4b10133c7669b146c', 'request_id': 'req-3d45c768-00f9-4340-86bb-1b4fdf460009'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.109838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.110027 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] The token cache is being invalidate because project f7e5e21696c14bf4b10133c7669b146c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:36.110128 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.110892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.111028 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-3d45c768-00f9-4340-86bb-1b4fdf460009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.111203 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.111500 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] PATH_INFO: `/v3/projects/59fd8eeb12dc4656929280a79d0a3d5d` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.113465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1371/2731] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:35 2022] DELETE /identity/v3/projects/f7e5e21696c14bf4b10133c7669b146c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:36.116227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59fd8eeb12dc4656929280a79d0a3d5d)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.117358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d93ea8-703b-4d93-bf76-fbd4d6c266ca None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.117654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d93ea8-703b-4d93-bf76-fbd4d6c266ca None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.117937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d93ea8-703b-4d93-bf76-fbd4d6c266ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.119656 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.129253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:36.137456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-b8d93ea8-703b-4d93-bf76-fbd4d6c266ca None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.138673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-b8d93ea8-703b-4d93-bf76-fbd4d6c266ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.148747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1372/2732] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:36.156261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8227f2-0f3e-47e8-9fa6-291d9036d428 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.156602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8227f2-0f3e-47e8-9fa6-291d9036d428 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.156892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8227f2-0f3e-47e8-9fa6-291d9036d428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.176824 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7f8227f2-0f3e-47e8-9fa6-291d9036d428 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.178024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7f8227f2-0f3e-47e8-9fa6-291d9036d428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.186033 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1373/2733] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:36.190051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8227f2-0f3e-47e8-9fa6-291d9036d428 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.199687 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8227f2-0f3e-47e8-9fa6-291d9036d428 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.204654 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.205298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.205582 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.206465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.207203 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:36.207551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.209138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.222831 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1374/2734] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:44:36.226890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.237298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb31427-7295-464a-a44f-27c9ae4c8ca9 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.239077 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59fd8eeb12dc4656929280a79d0a3d5d', 'request_id': 'req-6219cd22-f236-496c-836a-246d6f176193'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.247322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59fd8eeb12dc4656929280a79d0a3d5d', 'request_id': 'req-6219cd22-f236-496c-836a-246d6f176193'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.256502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59fd8eeb12dc4656929280a79d0a3d5d', 'request_id': 'req-6219cd22-f236-496c-836a-246d6f176193'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.296430 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.297094 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.297387 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.297670 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.298355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.300312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.311549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:36.325956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] The token cache is being invalidate because project 59fd8eeb12dc4656929280a79d0a3d5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:36.326322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.327070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6219cd22-f236-496c-836a-246d6f176193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:36.328687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1361/2735] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:36 2022] DELETE /identity/v3/projects/59fd8eeb12dc4656929280a79d0a3d5d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:36.334043 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7cab-50d5-4fb8-9e8c-482af7ca447c None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.334336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7cab-50d5-4fb8-9e8c-482af7ca447c None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.334621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7cab-50d5-4fb8-9e8c-482af7ca447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.353508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-3bde7cab-50d5-4fb8-9e8c-482af7ca447c None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.354652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3bde7cab-50d5-4fb8-9e8c-482af7ca447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.362921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1362/2736] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:36.370322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3c8b5-5a61-4246-bde1-edfe93ca2272 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.370615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3c8b5-5a61-4246-bde1-edfe93ca2272 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.370900 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3c8b5-5a61-4246-bde1-edfe93ca2272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.389180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-63c3c8b5-5a61-4246-bde1-edfe93ca2272 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.390371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-63c3c8b5-5a61-4246-bde1-edfe93ca2272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.399125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1363/2737] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:36.403319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3c8b5-5a61-4246-bde1-edfe93ca2272 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.408358 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1375/2738] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:36.413896 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3c8b5-5a61-4246-bde1-edfe93ca2272 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.414223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.415863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.475494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.476167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.476456 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.476738 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.477518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:36.477811 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.479424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.482043 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d07d-4537-439c-bef8-036ad8c72983 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.486864 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.491442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.491442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.491442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.491442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.491442 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.491950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1364/2739] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:44:36.495303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.506567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c33cb-7557-4b00-887e-96644d07977a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.512678 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.513700 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.517168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.517168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.517168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.517168 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.600042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1376/2740] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:36.606078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.614931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54f07e-77e7-40ee-b77a-626c2681646f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.618904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1365/2741] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:36.619888 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.620636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.621064 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.621353 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.622184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.622885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.623850 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.635550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1377/2742] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:36.640284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.651875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfd163-5158-4957-b3e8-abf1fddefc78 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.681345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:36.686059 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356dfeab-142d-47b3-8380-1442fec08da1 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.691792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.692468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.692753 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.693032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.693723 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.695672 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.711889 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.712570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.712861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.713149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.713884 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.715536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.725916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1378/2743] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:36.729742 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.739423 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99776014-dd58-4842-82b9-704afa1881fe admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.744092 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509645-bb60-492d-980c-0969b95561bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.744949 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509645-bb60-492d-980c-0969b95561bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.745244 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509645-bb60-492d-980c-0969b95561bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.745528 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509645-bb60-492d-980c-0969b95561bc admin admin] PATH_INFO: `/v3/projects/0c29c709972349119db4c40e22b122d4/users/656d1155b08740f7a36d809bf5f8be5b/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.760689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6509645-bb60-492d-980c-0969b95561bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c29c709972349119db4c40e22b122d4, user_id=656d1155b08740f7a36d809bf5f8be5b, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.763405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6509645-bb60-492d-980c-0969b95561bc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.794746 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1366/2744] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:36.798687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.808894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d53b7e-fa87-43ae-ab4f-04609fd4c8a4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.814240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24090cef-b238-420f-8282-806518f4240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.814899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24090cef-b238-420f-8282-806518f4240a admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.815883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24090cef-b238-420f-8282-806518f4240a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.816195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24090cef-b238-420f-8282-806518f4240a admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.816932 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24090cef-b238-420f-8282-806518f4240a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.818533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24090cef-b238-420f-8282-806518f4240a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.828737 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1367/2745] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:36.832341 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24090cef-b238-420f-8282-806518f4240a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.842082 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24090cef-b238-420f-8282-806518f4240a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.846637 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.848594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.848594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.848594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.848594 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.850058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.860249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1368/2746] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:36.864302 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.873719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ce25e-2ece-4d9f-8447-d810a3a44966 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:36.877426 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1379/2747] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:36 2022] PUT /identity/v3/projects/0c29c709972349119db4c40e22b122d4/users/656d1155b08740f7a36d809bf5f8be5b/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:36.887747 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.888038 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.888320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.915768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] MFA Rules not processed for user `656d1155b08740f7a36d809bf5f8be5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:36.917680 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:36.941893 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924fd7c-551f-40fe-8097-d415c9b5b42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:36.943224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924fd7c-551f-40fe-8097-d415c9b5b42d admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:36.943517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924fd7c-551f-40fe-8097-d415c9b5b42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:36.943800 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924fd7c-551f-40fe-8097-d415c9b5b42d admin admin] PATH_INFO: `/v3/projects/e44f6854b19b45638042b925e56dc90a/users/1cf0bcc6493345cba661d0aba5843b1e/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:36.948309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:36.959246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924fd7c-551f-40fe-8097-d415c9b5b42d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44f6854b19b45638042b925e56dc90a, user_id=1cf0bcc6493345cba661d0aba5843b1e, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:36.962182 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924fd7c-551f-40fe-8097-d415c9b5b42d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:36.983303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1380/2748] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:36.994990 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:36.996396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.053951 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7743ad-9630-4cfa-9645-8a1139d09a3b None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.057892 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1369/2749] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:36 2022] PUT /identity/v3/projects/e44f6854b19b45638042b925e56dc90a/users/1cf0bcc6493345cba661d0aba5843b1e/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:37.069106 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbdf1f-04f5-48a1-a599-619b0811a991 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.069377 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbdf1f-04f5-48a1-a599-619b0811a991 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.069662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbdf1f-04f5-48a1-a599-619b0811a991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.098092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-76dbdf1f-04f5-48a1-a599-619b0811a991 None None] MFA Rules not processed for user `1cf0bcc6493345cba661d0aba5843b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:37.099355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-76dbdf1f-04f5-48a1-a599-619b0811a991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.105303 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.105958 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.106385 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.106675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.162612 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.163999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.170815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1370/2750] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:37.174130 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1381/2751] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:37.175788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbdf1f-04f5-48a1-a599-619b0811a991 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.200177 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.222970 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:37.249180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbdf1f-04f5-48a1-a599-619b0811a991 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.256383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.257051 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.257338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.257618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.258316 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.260403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.261140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=100 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:37.279644 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c67194-e5e6-4dcf-85b8-7c7015b859fd service cinder] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.284903 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.285966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.286253 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.287729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] PATH_INFO: `/v3/users/656d1155b08740f7a36d809bf5f8be5b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.288986 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] RBAC: Authorizing `identity:delete_user(user_id=656d1155b08740f7a36d809bf5f8be5b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.290945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.366964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1371/2752] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:37 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:37.371222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.434817 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2292d32-e91c-4ea8-a3d9-997b3da76421 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.439395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.440056 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.441033 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.441317 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.442001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.443929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.483213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656d1155b08740f7a36d809bf5f8be5b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.490191 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656d1155b08740f7a36d809bf5f8be5b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.490882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656d1155b08740f7a36d809bf5f8be5b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.491525 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656d1155b08740f7a36d809bf5f8be5b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.504299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d58ff346-6240-40fb-8648-62f0a8a4952a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656d1155b08740f7a36d809bf5f8be5b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:37.514534 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1372/2753] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:37 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:37.519533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.521042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.533072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67d3fe-039b-4efd-ad67-37d41281e916 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.537969 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.538921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.538921 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.539262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.543303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.543303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.555558 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1373/2754] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:37.556852 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.560002 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1382/2755] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:37 2022] DELETE /identity/v3/users/656d1155b08740f7a36d809bf5f8be5b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:37.564587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.564885 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.565765 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.568356 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c2b0b-9116-4b61-b46c-ca69f2e0008b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.584899 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:37.586402 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.630102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.631046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.631379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.632202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] PATH_INFO: `/v3/projects/17a6c4c0b7bf4cf9af5c764352cc9f38/users/7e5c56f12e8146bc94068fb5973e83ce/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.648521 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a6c4c0b7bf4cf9af5c764352cc9f38, user_id=7e5c56f12e8146bc94068fb5973e83ce, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.651661 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.653551 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1383/2756] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:37.664452 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.665795 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.709658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1374/2757] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:37 2022] PUT /identity/v3/projects/17a6c4c0b7bf4cf9af5c764352cc9f38/users/7e5c56f12e8146bc94068fb5973e83ce/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:37.720375 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.740298 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b06db9-9b24-483a-b973-5a58bda531c4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.753610 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:37.755570 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:37.793058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6af4bf-d5de-47e0-88b4-7894707f2738 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.794892 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7c3e-4cc6-46bb-9df2-c8ad967d9799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.795577 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7c3e-4cc6-46bb-9df2-c8ad967d9799 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.795866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7c3e-4cc6-46bb-9df2-c8ad967d9799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.796201 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7c3e-4cc6-46bb-9df2-c8ad967d9799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.797688 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.798457 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.798745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.799055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.799832 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.801295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.807602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b7c3e-4cc6-46bb-9df2-c8ad967d9799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.809206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b7c3e-4cc6-46bb-9df2-c8ad967d9799 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.812158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1375/2758] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:37.816072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.819937 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1384/2759] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:44:37.826361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db935b6c-1e66-4041-9a90-395996a21f13 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:37.831870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:37.832841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.833200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.833482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] PATH_INFO: `/v3/projects/17a6c4c0b7bf4cf9af5c764352cc9f38/users/7e5c56f12e8146bc94068fb5973e83ce/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.837167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a6c4c0b7bf4cf9af5c764352cc9f38, user_id=7e5c56f12e8146bc94068fb5973e83ce, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:37.838416 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:37.849999 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1376/2760] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:37 2022] PUT /identity/v3/projects/17a6c4c0b7bf4cf9af5c764352cc9f38/users/7e5c56f12e8146bc94068fb5973e83ce/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:37.861615 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:37.862068 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:37.862762 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:37.896483 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] MFA Rules not processed for user `7e5c56f12e8146bc94068fb5973e83ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:37.897696 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:37.974872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1385/2761] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:37.988880 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:37.990839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:38.059425 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184e9f0-c84d-4880-b14f-1d5a67150b47 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.063432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:38.064084 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.064371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.064647 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.074876 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:38.076455 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:38.087517 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1377/2762] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:38.229432 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.232052 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:38.299190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf8a0b-1eb5-4b87-bed2-6f6af3f3f53d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.304373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:38.305228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:38.305511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:38.305785 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] PATH_INFO: `/v3/projects/0c29c709972349119db4c40e22b122d4` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:38.306954 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c29c709972349119db4c40e22b122d4)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:38.308887 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:38.422983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c29c709972349119db4c40e22b122d4', 'request_id': 'req-7817194a-2cb9-47b1-9a66-5802ed61c1bc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:38.429451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c29c709972349119db4c40e22b122d4', 'request_id': 'req-7817194a-2cb9-47b1-9a66-5802ed61c1bc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:38.438849 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c29c709972349119db4c40e22b122d4', 'request_id': 'req-7817194a-2cb9-47b1-9a66-5802ed61c1bc'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:38.462890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:38.519523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] The token cache is being invalidate because project 0c29c709972349119db4c40e22b122d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:38.520124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:38.520454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7817194a-2cb9-47b1-9a66-5802ed61c1bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:38.521981 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1386/2763] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:38 2022] DELETE /identity/v3/projects/0c29c709972349119db4c40e22b122d4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:38.932295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:38.943040 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1151f-896e-49b9-b763-88a0e763508a service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:38.999502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.000369 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.000654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.000933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] PATH_INFO: `/v3/users/1cf0bcc6493345cba661d0aba5843b1e` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.002097 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf0bcc6493345cba661d0aba5843b1e)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.004227 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.167759 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf0bcc6493345cba661d0aba5843b1e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:39.168440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf0bcc6493345cba661d0aba5843b1e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:39.170379 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:39.175642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf0bcc6493345cba661d0aba5843b1e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:39.176271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf0bcc6493345cba661d0aba5843b1e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:39.228309 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf0bcc6493345cba661d0aba5843b1e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:39.246368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1378/2764] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:38 2022] DELETE /identity/v3/users/1cf0bcc6493345cba661d0aba5843b1e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:39.250901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.251216 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.251504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.270523 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:39.271752 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:39.329718 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1387/2765] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:39.340030 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.341464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:39.399840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027fc7a-7a35-4536-bf07-ce7c4f3e27e2 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.403421 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.404062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.404347 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.404625 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.413962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.415480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.425006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1379/2766] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:39.795129 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:39.796638 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:39.810841 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c09ec-85d6-489c-ac6e-c2a60230ce19 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:39.816170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:39.817135 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:39.817417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:39.817694 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] PATH_INFO: `/v3/projects/e44f6854b19b45638042b925e56dc90a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:39.818882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44f6854b19b45638042b925e56dc90a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:39.820944 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:39.931425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44f6854b19b45638042b925e56dc90a', 'request_id': 'req-6aa0424e-f5be-4103-8f12-d9cfb069fafa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:39.936978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44f6854b19b45638042b925e56dc90a', 'request_id': 'req-6aa0424e-f5be-4103-8f12-d9cfb069fafa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:39.945331 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44f6854b19b45638042b925e56dc90a', 'request_id': 'req-6aa0424e-f5be-4103-8f12-d9cfb069fafa'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.032401 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] The token cache is being invalidate because project e44f6854b19b45638042b925e56dc90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:40.032689 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.033301 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.034907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1388/2767] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:39 2022] DELETE /identity/v3/projects/e44f6854b19b45638042b925e56dc90a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:40.038855 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.048481 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8499309-e227-4927-8936-a1c940fbf06a service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.107156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.108045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.108331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.108616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] PATH_INFO: `/v3/users/7e5c56f12e8146bc94068fb5973e83ce` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.109785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5c56f12e8146bc94068fb5973e83ce)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.111740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.282116 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5c56f12e8146bc94068fb5973e83ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.282765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5c56f12e8146bc94068fb5973e83ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.290458 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5c56f12e8146bc94068fb5973e83ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.291094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5c56f12e8146bc94068fb5973e83ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.353673 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-ed428bd6-b3a9-4413-a771-8c84d35ae9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5c56f12e8146bc94068fb5973e83ce'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:40.361075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:40.372721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1380/2768] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:40 2022] DELETE /identity/v3/users/7e5c56f12e8146bc94068fb5973e83ce => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:40.812428 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:40.813974 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:40.828555 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0424e-f5be-4103-8f12-d9cfb069fafa admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:40.888926 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:40.893124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:40.893124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:40.893124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] PATH_INFO: `/v3/projects/17a6c4c0b7bf4cf9af5c764352cc9f38` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:40.893124 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a6c4c0b7bf4cf9af5c764352cc9f38)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:40.894622 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:40.896267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:41.006498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a6c4c0b7bf4cf9af5c764352cc9f38', 'request_id': 'req-57d0d0ba-2deb-465f-8ba4-29d214d23a92'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:41.012645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a6c4c0b7bf4cf9af5c764352cc9f38', 'request_id': 'req-57d0d0ba-2deb-465f-8ba4-29d214d23a92'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:41.019632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a6c4c0b7bf4cf9af5c764352cc9f38', 'request_id': 'req-57d0d0ba-2deb-465f-8ba4-29d214d23a92'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:41.092977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] The token cache is being invalidate because project 17a6c4c0b7bf4cf9af5c764352cc9f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:41.093194 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:41.093891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-57d0d0ba-2deb-465f-8ba4-29d214d23a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:41.095822 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1389/2769] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:40 2022] DELETE /identity/v3/projects/17a6c4c0b7bf4cf9af5c764352cc9f38 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:41.100502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc2cb6-6621-495b-b9e7-eb75403a2be8 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.101306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc2cb6-6621-495b-b9e7-eb75403a2be8 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.101825 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc2cb6-6621-495b-b9e7-eb75403a2be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.122000 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-53cc2cb6-6621-495b-b9e7-eb75403a2be8 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:41.123567 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-53cc2cb6-6621-495b-b9e7-eb75403a2be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.189339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1381/2770] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:41.197991 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfe17d-f6e4-4ea5-b29c-e32689e9252d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.198284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfe17d-f6e4-4ea5-b29c-e32689e9252d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.198566 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfe17d-f6e4-4ea5-b29c-e32689e9252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.225181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-19cfe17d-f6e4-4ea5-b29c-e32689e9252d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:41.226379 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-19cfe17d-f6e4-4ea5-b29c-e32689e9252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.234596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1390/2771] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:41.239498 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc2cb6-6621-495b-b9e7-eb75403a2be8 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.249816 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc2cb6-6621-495b-b9e7-eb75403a2be8 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.255612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.256351 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.256640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.256920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.257679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:41.257984 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.259619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.272463 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1382/2772] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:41.276810 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfe17d-f6e4-4ea5-b29c-e32689e9252d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.286420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfe17d-f6e4-4ea5-b29c-e32689e9252d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.291266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.291924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.292211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.292490 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.293182 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.295079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.417418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1391/2773] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:41.421544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.432376 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e530be5-56d2-4729-a28d-3242cc630347 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.491004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.491705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.491995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.492277 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.492977 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.494901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.599765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1383/2774] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:41.603931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.614854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192074c-902b-4500-b80d-fd10f4b0ac0f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.619526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.620181 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.620468 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.620749 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.621607 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.623279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.634224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1392/2775] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:41.644107 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.654331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46efe9-f95f-45f3-b70c-b20c6c946b9a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.659531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90e5e7-3de2-4260-873a-01244ecaa1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.660231 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90e5e7-3de2-4260-873a-01244ecaa1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.665719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90e5e7-3de2-4260-873a-01244ecaa1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.665719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90e5e7-3de2-4260-873a-01244ecaa1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.665719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90e5e7-3de2-4260-873a-01244ecaa1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.665719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90e5e7-3de2-4260-873a-01244ecaa1e1 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.673122 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1384/2776] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:41.677693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:41.687738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90647c96-8ba1-42fd-a82f-de9e79cdc2b6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:41.692947 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:41.694456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.694456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.694456 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] PATH_INFO: `/v3/projects/25b88bd45acb4407bdf26ae43302798f/users/c5b1d6a61eb34329a916a593e755071a/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.709425 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b88bd45acb4407bdf26ae43302798f, user_id=c5b1d6a61eb34329a916a593e755071a, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:41.712455 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:41.805533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1393/2777] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:41 2022] PUT /identity/v3/projects/25b88bd45acb4407bdf26ae43302798f/users/c5b1d6a61eb34329a916a593e755071a/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:41.864440 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e05ec-a4f6-4617-883b-3763a67d0899 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:41.864838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e05ec-a4f6-4617-883b-3763a67d0899 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:41.865225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e05ec-a4f6-4617-883b-3763a67d0899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:41.892125 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d95e05ec-a4f6-4617-883b-3763a67d0899 None None] MFA Rules not processed for user `c5b1d6a61eb34329a916a593e755071a`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:41.893332 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d95e05ec-a4f6-4617-883b-3763a67d0899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:41.937185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:41.959523 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1385/2778] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:41.964453 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.030388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92989d-8949-4150-aab2-528160672a68 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.035433 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.036098 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.036390 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.036704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.037396 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.039361 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.115540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1394/2779] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:41 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:42.119669 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e05ec-a4f6-4617-883b-3763a67d0899 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.130410 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e05ec-a4f6-4617-883b-3763a67d0899 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.188021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.188621 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.188913 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.189192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.190018 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.192015 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.276751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1386/2780] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:42 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:42.280735 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.292369 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa94d8-b258-40fb-a2be-8c70cabbb310 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.297138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.297814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.298103 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.298404 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.299167 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.300979 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.306414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:42.312316 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1395/2781] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:42.318207 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.328729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e402a0-c5da-447f-9971-62abe38d57e5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.334094 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.337910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.337910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.337910 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] PATH_INFO: `/v3/projects/3dfb36a4a50345cdb10cc728c03ef166/users/26ed332898474186a384555d7d7fe05a/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.352830 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfb36a4a50345cdb10cc728c03ef166, user_id=26ed332898474186a384555d7d7fe05a, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.355551 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.410959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1387/2782] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:42 2022] PUT /identity/v3/projects/3dfb36a4a50345cdb10cc728c03ef166/users/26ed332898474186a384555d7d7fe05a/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:42.414912 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.425266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a932df-cf4f-4ecd-8ec6-0e33cb2da9b2 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.485256 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edfb7f-5ed2-4d1d-82af-64367c90e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.485938 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edfb7f-5ed2-4d1d-82af-64367c90e44d admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.486230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edfb7f-5ed2-4d1d-82af-64367c90e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.486545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edfb7f-5ed2-4d1d-82af-64367c90e44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.487333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edfb7f-5ed2-4d1d-82af-64367c90e44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.489076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edfb7f-5ed2-4d1d-82af-64367c90e44d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.499381 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1396/2783] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:42.504395 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.514642 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1d4f9-4f7d-4a05-b31b-b5c3cec5f876 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.520061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.520934 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.521218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.521496 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] PATH_INFO: `/v3/projects/3dfb36a4a50345cdb10cc728c03ef166/users/26ed332898474186a384555d7d7fe05a/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.523497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfb36a4a50345cdb10cc728c03ef166, user_id=26ed332898474186a384555d7d7fe05a, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.526118 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.536261 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1388/2784] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:42 2022] PUT /identity/v3/projects/3dfb36a4a50345cdb10cc728c03ef166/users/26ed332898474186a384555d7d7fe05a/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:42.546333 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb52dd-70cf-48c4-aa64-6e8c9b96bc9a None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.546633 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb52dd-70cf-48c4-aa64-6e8c9b96bc9a None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.546918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb52dd-70cf-48c4-aa64-6e8c9b96bc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.574065 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-35cb52dd-70cf-48c4-aa64-6e8c9b96bc9a None None] MFA Rules not processed for user `26ed332898474186a384555d7d7fe05a`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:42.575306 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-35cb52dd-70cf-48c4-aa64-6e8c9b96bc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:42.646746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1397/2785] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:42.657863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:42.660167 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:42.721840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fdce3-a197-42ad-a0a2-0fba1f9f5948 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:42.725386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c8819-c736-403d-91a0-06487472f6b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:42.726048 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c8819-c736-403d-91a0-06487472f6b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:42.726322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c8819-c736-403d-91a0-06487472f6b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:42.726603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c8819-c736-403d-91a0-06487472f6b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:42.735530 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c8819-c736-403d-91a0-06487472f6b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:42.737022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c8819-c736-403d-91a0-06487472f6b2 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:42.746889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1389/2786] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:42 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:45.661013 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb52dd-70cf-48c4-aa64-6e8c9b96bc9a None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:45.722616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb52dd-70cf-48c4-aa64-6e8c9b96bc9a None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:45.727117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:45.728030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:45.728312 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:45.728584 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] PATH_INFO: `/v3/users/c5b1d6a61eb34329a916a593e755071a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:45.729829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b1d6a61eb34329a916a593e755071a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:45.731786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:45.755711 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:45.933581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b1d6a61eb34329a916a593e755071a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:45.941015 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b1d6a61eb34329a916a593e755071a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:45.941842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b1d6a61eb34329a916a593e755071a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:45.942447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b1d6a61eb34329a916a593e755071a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:45.954447 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b1d6a61eb34329a916a593e755071a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:46.090112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1398/2787] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:45 2022] DELETE /identity/v3/users/c5b1d6a61eb34329a916a593e755071a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:46.094532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:46.095044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:46.095158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:46.112616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:46.113982 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:46.149402 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:46.177247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1390/2788] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:46.186961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:46.188406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:46.256102 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964c0e0-d1b2-4248-9b8a-0b9a8671c633 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:46.259801 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:46.260450 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:46.260741 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:46.261071 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:46.270427 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:46.271897 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:46.281294 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1399/2789] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:46.680915 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:46.684615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:46.698654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db28a0-49de-455d-9ac1-4ca337313be1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:46.703568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:46.704461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:46.704747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:46.705055 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] PATH_INFO: `/v3/projects/25b88bd45acb4407bdf26ae43302798f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:46.707136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b88bd45acb4407bdf26ae43302798f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:46.713136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:46.824680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b88bd45acb4407bdf26ae43302798f', 'request_id': 'req-149ebf6c-899a-42e6-91b3-799875b3ab6d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:46.831566 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b88bd45acb4407bdf26ae43302798f', 'request_id': 'req-149ebf6c-899a-42e6-91b3-799875b3ab6d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:46.838210 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b88bd45acb4407bdf26ae43302798f', 'request_id': 'req-149ebf6c-899a-42e6-91b3-799875b3ab6d'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:46.930994 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] The token cache is being invalidate because project 25b88bd45acb4407bdf26ae43302798f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:46.931338 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:46.931963 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:46.933510 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1391/2790] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:46 2022] DELETE /identity/v3/projects/25b88bd45acb4407bdf26ae43302798f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:46.937543 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:46.948062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f30f94-031b-4c6b-a696-b75d109a85b6 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:47.005925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:47.006806 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:47.007093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:47.007403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] PATH_INFO: `/v3/users/26ed332898474186a384555d7d7fe05a` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:47.008617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ed332898474186a384555d7d7fe05a)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:47.010602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:47.227895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:47.235851 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ed332898474186a384555d7d7fe05a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.243305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ed332898474186a384555d7d7fe05a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.244079 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ed332898474186a384555d7d7fe05a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.244698 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ed332898474186a384555d7d7fe05a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.260067 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-4fb668b8-ff6b-41a3-a26c-b8f81032f726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ed332898474186a384555d7d7fe05a'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.348397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1400/2791] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:46 2022] DELETE /identity/v3/users/26ed332898474186a384555d7d7fe05a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:47.761234 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:47.762846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:47.777065 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ebf6c-899a-42e6-91b3-799875b3ab6d admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:47.849798 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:47.850840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:47.851123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:47.851430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] PATH_INFO: `/v3/projects/3dfb36a4a50345cdb10cc728c03ef166` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:47.852622 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfb36a4a50345cdb10cc728c03ef166)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:47.854535 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:47.966752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfb36a4a50345cdb10cc728c03ef166', 'request_id': 'req-7047554a-8330-4498-aa44-d7cd29093589'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.973068 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfb36a4a50345cdb10cc728c03ef166', 'request_id': 'req-7047554a-8330-4498-aa44-d7cd29093589'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.980218 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfb36a4a50345cdb10cc728c03ef166', 'request_id': 'req-7047554a-8330-4498-aa44-d7cd29093589'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:47.982827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:48.052287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] The token cache is being invalidate because project 3dfb36a4a50345cdb10cc728c03ef166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:48.052649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:48.053359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-7047554a-8330-4498-aa44-d7cd29093589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:48.055221 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1392/2792] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:47 2022] DELETE /identity/v3/projects/3dfb36a4a50345cdb10cc728c03ef166 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:48.060195 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd426-f0f5-4812-8d67-52c1cb2ae9a2 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.060692 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd426-f0f5-4812-8d67-52c1cb2ae9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.060814 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd426-f0f5-4812-8d67-52c1cb2ae9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.078251 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e2cbd426-f0f5-4812-8d67-52c1cb2ae9a2 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:48.079545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e2cbd426-f0f5-4812-8d67-52c1cb2ae9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:48.142022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1401/2793] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:44:48.150531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f443-7bbe-4a03-81dc-ff39fffade55 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.150840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f443-7bbe-4a03-81dc-ff39fffade55 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.151120 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f443-7bbe-4a03-81dc-ff39fffade55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.171405 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9e33f443-7bbe-4a03-81dc-ff39fffade55 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:48.172633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9e33f443-7bbe-4a03-81dc-ff39fffade55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:48.181322 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1393/2794] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:48.185804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd426-f0f5-4812-8d67-52c1cb2ae9a2 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.197066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd426-f0f5-4812-8d67-52c1cb2ae9a2 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.201825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.202496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.202790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.203078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.203854 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:48.204172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.207929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:48.220703 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1402/2795] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:44:48.226199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f443-7bbe-4a03-81dc-ff39fffade55 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.235634 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f443-7bbe-4a03-81dc-ff39fffade55 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.241464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.242130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.242474 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.242761 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.243518 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.245442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:48.331744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1394/2796] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:48 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:48.337624 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.343329 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deecc5a-740d-4861-9904-2fa3650b59bc admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.405366 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.406314 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.406608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.407053 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.407969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.410112 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:48.475139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1403/2797] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:48 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:48.479398 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.490840 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40874e4a-7f2f-4806-a069-3cafe73a3caa admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.495575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.496229 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.496514 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.496799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.497531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.499127 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:48.513889 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1395/2798] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:48.519235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.529449 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc55c1-f644-46c5-872e-7322ee6b4d5c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.534332 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245f44c-249e-4295-be9e-885d32890365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.534997 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245f44c-249e-4295-be9e-885d32890365 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.535326 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245f44c-249e-4295-be9e-885d32890365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.535616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245f44c-249e-4295-be9e-885d32890365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.536352 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245f44c-249e-4295-be9e-885d32890365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.538134 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245f44c-249e-4295-be9e-885d32890365 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:48.549224 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1404/2799] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:48.553436 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.563378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f8d0-d391-4a61-b1e9-1f30389f4933 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.568806 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.569714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.570002 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.570281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] PATH_INFO: `/v3/projects/d47b7210296640c38865242ed0251fe6/users/00aa63eb00c744e98e12d0d6aeb343f1/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.586075 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47b7210296640c38865242ed0251fe6, user_id=00aa63eb00c744e98e12d0d6aeb343f1, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.588696 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:48.663255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1396/2800] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:48 2022] PUT /identity/v3/projects/d47b7210296640c38865242ed0251fe6/users/00aa63eb00c744e98e12d0d6aeb343f1/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:48.675085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c239e3f-5d10-4ac1-8aad-e23fa165366f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.675578 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c239e3f-5d10-4ac1-8aad-e23fa165366f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.675969 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c239e3f-5d10-4ac1-8aad-e23fa165366f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.704045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-6c239e3f-5d10-4ac1-8aad-e23fa165366f None None] MFA Rules not processed for user `00aa63eb00c744e98e12d0d6aeb343f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:48.705280 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-6c239e3f-5d10-4ac1-8aad-e23fa165366f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:48.714782 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:48.774034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1405/2801] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:48.778775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.845032 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d1d50-21dd-42b8-9950-45d2e23425e7 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.849964 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.850641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.850952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.851271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.852011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.854089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:48.917136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1397/2802] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:48 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:48.921225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c239e3f-5d10-4ac1-8aad-e23fa165366f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:48.932942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c239e3f-5d10-4ac1-8aad-e23fa165366f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:48.995003 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:48.995714 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:48.996007 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:48.996287 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:48.996999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:48.998929 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:49.075857 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1406/2803] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:48 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:49.079895 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:49.090662 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe46e8-468e-46b6-abdb-f46b0853f3f4 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:49.097013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:49.097652 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:49.098482 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:49.098785 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:49.099586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:49.101253 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:49.112885 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1398/2804] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:49.117497 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:49.127998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ea56c-65df-46c6-8220-7e40f4f2edfb admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:49.133055 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:49.133972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:49.134260 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:49.134575 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] PATH_INFO: `/v3/projects/b3f477f754de48dc959656d6b47dbd27/users/057b5deeffe744719065dd1f4517fd6e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:49.150913 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f477f754de48dc959656d6b47dbd27, user_id=057b5deeffe744719065dd1f4517fd6e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:49.153637 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:49.205983 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1407/2805] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:49 2022] PUT /identity/v3/projects/b3f477f754de48dc959656d6b47dbd27/users/057b5deeffe744719065dd1f4517fd6e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:49.210504 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:49.220645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3cf9c-ecc4-41b2-ae16-bfb2ea304ccb admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:49.279013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ff5e6-2754-4570-92d4-b6ded9b4bf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:49.279730 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ff5e6-2754-4570-92d4-b6ded9b4bf2d admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:49.280021 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ff5e6-2754-4570-92d4-b6ded9b4bf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:49.280306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ff5e6-2754-4570-92d4-b6ded9b4bf2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:49.281045 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ff5e6-2754-4570-92d4-b6ded9b4bf2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:49.282657 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ff5e6-2754-4570-92d4-b6ded9b4bf2d admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:49.288711 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:49.296061 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1399/2806] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:44:49.298178 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:49.311143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309a95d-c75a-44e3-a359-e07f2996299a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:49.316169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:49.317420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:49.317420 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:49.317729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] PATH_INFO: `/v3/projects/b3f477f754de48dc959656d6b47dbd27/users/057b5deeffe744719065dd1f4517fd6e/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:49.319759 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f477f754de48dc959656d6b47dbd27, user_id=057b5deeffe744719065dd1f4517fd6e, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:49.322510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:49.333489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1408/2807] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:49 2022] PUT /identity/v3/projects/b3f477f754de48dc959656d6b47dbd27/users/057b5deeffe744719065dd1f4517fd6e/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:49.344382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828acf1-0a98-4acf-bc3f-0f69d44e66c1 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:49.344679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828acf1-0a98-4acf-bc3f-0f69d44e66c1 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:49.344965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828acf1-0a98-4acf-bc3f-0f69d44e66c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:49.373695 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d828acf1-0a98-4acf-bc3f-0f69d44e66c1 None None] MFA Rules not processed for user `057b5deeffe744719065dd1f4517fd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:49.374862 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d828acf1-0a98-4acf-bc3f-0f69d44e66c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:49.446254 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1400/2808] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:49.457931 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:49.459701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:49.525939 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d414d8-cd51-46d9-ad5c-2ee4b82d82b0 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:49.529928 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ab3d9-f4e2-45f3-9458-5ab9fc1f3b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:49.530608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ab3d9-f4e2-45f3-9458-5ab9fc1f3b62 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:49.530895 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ab3d9-f4e2-45f3-9458-5ab9fc1f3b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:49.531262 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ab3d9-f4e2-45f3-9458-5ab9fc1f3b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:49.541403 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ab3d9-f4e2-45f3-9458-5ab9fc1f3b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:49.542968 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ab3d9-f4e2-45f3-9458-5ab9fc1f3b62 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:49.554754 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1409/2809] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:49.584071 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828acf1-0a98-4acf-bc3f-0f69d44e66c1 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:49.650220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828acf1-0a98-4acf-bc3f-0f69d44e66c1 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:49.655159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:49.656069 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:49.656364 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:49.656645 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] PATH_INFO: `/v3/users/00aa63eb00c744e98e12d0d6aeb343f1` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:49.657861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00aa63eb00c744e98e12d0d6aeb343f1)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:49.659959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:49.852238 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00aa63eb00c744e98e12d0d6aeb343f1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:49.852935 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00aa63eb00c744e98e12d0d6aeb343f1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:49.860146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00aa63eb00c744e98e12d0d6aeb343f1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:49.860781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00aa63eb00c744e98e12d0d6aeb343f1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:49.982086 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00aa63eb00c744e98e12d0d6aeb343f1'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:50.001853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1401/2810] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:49 2022] DELETE /identity/v3/users/00aa63eb00c744e98e12d0d6aeb343f1 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:50.006480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:50.006776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:50.007089 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:50.013918 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:50.025321 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:50.026518 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:50.088650 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1410/2811] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:50.098430 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:50.099860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:50.107702 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:50.161161 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e9edc-34d0-4d60-b2fb-07ebd7d69701 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:50.165709 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:50.166355 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:50.166640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:50.166918 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:50.176495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:50.177863 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:50.189195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1402/2812] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:50.629744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:50.631477 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:50.646190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c078a-fe63-4133-aa48-7bf032ba82db None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:50.651821 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:50.652599 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:50.652883 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:50.653161 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] PATH_INFO: `/v3/projects/d47b7210296640c38865242ed0251fe6` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:50.654995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47b7210296640c38865242ed0251fe6)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:50.657005 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:50.779018 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47b7210296640c38865242ed0251fe6', 'request_id': 'req-f8a02e56-21bb-4acc-84f8-4abc4b22480d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:50.784625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47b7210296640c38865242ed0251fe6', 'request_id': 'req-f8a02e56-21bb-4acc-84f8-4abc4b22480d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:50.791090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47b7210296640c38865242ed0251fe6', 'request_id': 'req-f8a02e56-21bb-4acc-84f8-4abc4b22480d'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:50.850547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] The token cache is being invalidate because project d47b7210296640c38865242ed0251fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:50.850846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:50.851567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:50.853139 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1411/2813] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:50 2022] DELETE /identity/v3/projects/d47b7210296640c38865242ed0251fe6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:50.857080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:50.867467 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22742b7-8a0f-443b-b90a-41a06ac5182a service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:50.928226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:50.929721 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:50.930011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:50.930295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] PATH_INFO: `/v3/users/057b5deeffe744719065dd1f4517fd6e` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:50.931595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057b5deeffe744719065dd1f4517fd6e)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:50.933532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:51.174536 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057b5deeffe744719065dd1f4517fd6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.175378 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057b5deeffe744719065dd1f4517fd6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.183114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057b5deeffe744719065dd1f4517fd6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.183788 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057b5deeffe744719065dd1f4517fd6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.268403 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-e73ce732-d365-4da2-894b-c8c3221a6bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057b5deeffe744719065dd1f4517fd6e'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.287329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1403/2814] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:50 2022] DELETE /identity/v3/users/057b5deeffe744719065dd1f4517fd6e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:51.698406 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:51.700672 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:51.714923 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a02e56-21bb-4acc-84f8-4abc4b22480d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:51.776676 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:51.777880 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:51.778220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:51.778571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] PATH_INFO: `/v3/projects/b3f477f754de48dc959656d6b47dbd27` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:51.780120 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f477f754de48dc959656d6b47dbd27)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:51.782346 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:51.873307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:51.886302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f477f754de48dc959656d6b47dbd27', 'request_id': 'req-a62c3d85-3275-4413-831a-4069092ad3dd'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.892439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f477f754de48dc959656d6b47dbd27', 'request_id': 'req-a62c3d85-3275-4413-831a-4069092ad3dd'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.899945 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f477f754de48dc959656d6b47dbd27', 'request_id': 'req-a62c3d85-3275-4413-831a-4069092ad3dd'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.987625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] The token cache is being invalidate because project b3f477f754de48dc959656d6b47dbd27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:51.987916 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.988583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-a62c3d85-3275-4413-831a-4069092ad3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:51.990267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1412/2815] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:51 2022] DELETE /identity/v3/projects/b3f477f754de48dc959656d6b47dbd27 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:51.995368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dff4a8-0a26-4ccf-bf37-5c6402373e7a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:51.995867 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dff4a8-0a26-4ccf-bf37-5c6402373e7a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:51.995959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dff4a8-0a26-4ccf-bf37-5c6402373e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.014553 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-f1dff4a8-0a26-4ccf-bf37-5c6402373e7a None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:52.015793 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-f1dff4a8-0a26-4ccf-bf37-5c6402373e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:52.037590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:52.079926 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1404/2816] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:52.088480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b4aa5-bbd6-4cce-aa2a-dffe9e80aba3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.088882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b4aa5-bbd6-4cce-aa2a-dffe9e80aba3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.089169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b4aa5-bbd6-4cce-aa2a-dffe9e80aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.109266 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f21b4aa5-bbd6-4cce-aa2a-dffe9e80aba3 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:52.110494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f21b4aa5-bbd6-4cce-aa2a-dffe9e80aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:52.119394 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1413/2817] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:52.124145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dff4a8-0a26-4ccf-bf37-5c6402373e7a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.134603 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dff4a8-0a26-4ccf-bf37-5c6402373e7a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.139268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.139919 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.140208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.140488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.141220 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:52.141533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.143132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.156017 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1405/2818] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:52.166113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b4aa5-bbd6-4cce-aa2a-dffe9e80aba3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.170547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b4aa5-bbd6-4cce-aa2a-dffe9e80aba3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.175817 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.176554 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.176842 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.177183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.178012 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.180188 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.305072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1414/2819] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:52 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:52.309371 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.320641 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048c3da-52c7-4b5c-90cb-ad50b72ff5b9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.379851 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.380444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.380735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.381022 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.381717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.384268 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.526392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1406/2820] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:52 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:52.531117 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.543564 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbcfee-360c-4270-896d-8e1f8b69042d admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.548189 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.548861 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.549147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.549445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.550220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.551859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.570370 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1415/2821] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:52.573179 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.583462 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0715e83-9c6e-4b41-b920-d4c1569607ba admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.588576 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8828718-5d0d-480a-8618-75725ecf313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.589252 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828718-5d0d-480a-8618-75725ecf313b admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.589542 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828718-5d0d-480a-8618-75725ecf313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.589827 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828718-5d0d-480a-8618-75725ecf313b admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.590569 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8828718-5d0d-480a-8618-75725ecf313b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.592204 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8828718-5d0d-480a-8618-75725ecf313b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.602933 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1407/2822] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:52.606936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.618454 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e62dfa-6c5a-4740-b040-6ba8a5a1936c admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.623162 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.624078 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.624388 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.624668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] PATH_INFO: `/v3/projects/aeb5c7b361be4ee98c893f096b150b85/users/109962dc7ec943b9aebc66d7ce6a26a5/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.642932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb5c7b361be4ee98c893f096b150b85, user_id=109962dc7ec943b9aebc66d7ce6a26a5, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.645639 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:52.707517 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1416/2823] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:52 2022] PUT /identity/v3/projects/aeb5c7b361be4ee98c893f096b150b85/users/109962dc7ec943b9aebc66d7ce6a26a5/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:44:52.719386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a7ac6-82fa-46c1-9eaa-024f3f4d876d None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.719729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a7ac6-82fa-46c1-9eaa-024f3f4d876d None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.720041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a7ac6-82fa-46c1-9eaa-024f3f4d876d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.747501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-c16a7ac6-82fa-46c1-9eaa-024f3f4d876d None None] MFA Rules not processed for user `109962dc7ec943b9aebc66d7ce6a26a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:52.748729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-c16a7ac6-82fa-46c1-9eaa-024f3f4d876d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:52.816288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1408/2824] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:52.820890 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:52.886465 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2ab10-409e-49b0-ae38-f1d38fc3c233 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:52.891307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:52.891977 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.892269 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.892554 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.893315 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:52.895237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.024235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1417/2825] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:52 2022] POST /identity/v3/projects => generated 389 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:53.028470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a7ac6-82fa-46c1-9eaa-024f3f4d876d None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.040070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a7ac6-82fa-46c1-9eaa-024f3f4d876d None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.059617 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:53.098362 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.099046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.099360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.099638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.100331 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.102838 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.212192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1409/2826] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:53 2022] POST /identity/v3/users => generated 301 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:53.216350 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.227823 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c1fe4-2764-489e-9cc5-8e33f21ea04a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.232539 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4907974-896c-4000-b601-84303757a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.233267 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4907974-896c-4000-b601-84303757a34f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.233546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4907974-896c-4000-b601-84303757a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.233827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4907974-896c-4000-b601-84303757a34f admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.234572 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4907974-896c-4000-b601-84303757a34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.236211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4907974-896c-4000-b601-84303757a34f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.247305 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1418/2827] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:53.251487 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.263609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a514d-a4a9-4acd-8a6f-fa04e28a9352 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.270187 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.271081 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.271394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.271680 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] PATH_INFO: `/v3/projects/bbf2e72f01384082921895e5615a3737/users/5acb672014034061ac9864a1445fe2ee/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.289276 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf2e72f01384082921895e5615a3737, user_id=5acb672014034061ac9864a1445fe2ee, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.292011 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.366601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1410/2828] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:53 2022] PUT /identity/v3/projects/bbf2e72f01384082921895e5615a3737/users/5acb672014034061ac9864a1445fe2ee/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:53.370691 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4907974-896c-4000-b601-84303757a34f admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.380905 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4907974-896c-4000-b601-84303757a34f admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.407590 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:53.438498 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6df76d-250f-4b2c-a237-1038509c8ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.439172 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6df76d-250f-4b2c-a237-1038509c8ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.439494 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6df76d-250f-4b2c-a237-1038509c8ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.439779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6df76d-250f-4b2c-a237-1038509c8ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.440549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6df76d-250f-4b2c-a237-1038509c8ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.442154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6df76d-250f-4b2c-a237-1038509c8ed7 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.453942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1419/2829] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:53.458129 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.469751 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3e29e-2dea-469b-9b26-3103846025f8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.475337 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a832370-defa-4217-aef7-544493315253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.476843 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a832370-defa-4217-aef7-544493315253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.477130 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a832370-defa-4217-aef7-544493315253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.477409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a832370-defa-4217-aef7-544493315253 admin admin] PATH_INFO: `/v3/projects/bbf2e72f01384082921895e5615a3737/users/5acb672014034061ac9864a1445fe2ee/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.479519 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a832370-defa-4217-aef7-544493315253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf2e72f01384082921895e5615a3737, user_id=5acb672014034061ac9864a1445fe2ee, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.482177 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a832370-defa-4217-aef7-544493315253 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.492831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1411/2830] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:53 2022] PUT /identity/v3/projects/bbf2e72f01384082921895e5615a3737/users/5acb672014034061ac9864a1445fe2ee/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:53.503261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b04e-f3ea-4e47-a3e7-34a65e059171 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.503562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b04e-f3ea-4e47-a3e7-34a65e059171 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.503853 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b04e-f3ea-4e47-a3e7-34a65e059171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.532249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3ab7b04e-f3ea-4e47-a3e7-34a65e059171 None None] MFA Rules not processed for user `5acb672014034061ac9864a1445fe2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:53.533481 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3ab7b04e-f3ea-4e47-a3e7-34a65e059171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.608500 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1420/2831] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:53.619616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a832370-defa-4217-aef7-544493315253 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.621358 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-8a832370-defa-4217-aef7-544493315253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:53.687894 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a832370-defa-4217-aef7-544493315253 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.691675 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5d3d9-966f-4dd0-b877-c0b8004cdaf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.692328 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d3d9-966f-4dd0-b877-c0b8004cdaf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.692616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d3d9-966f-4dd0-b877-c0b8004cdaf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.692898 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d3d9-966f-4dd0-b877-c0b8004cdaf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.702904 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5d3d9-966f-4dd0-b877-c0b8004cdaf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.704386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5d3d9-966f-4dd0-b877-c0b8004cdaf5 service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.715533 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1412/2832] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:53.849970 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b04e-f3ea-4e47-a3e7-34a65e059171 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.919964 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b04e-f3ea-4e47-a3e7-34a65e059171 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.924254 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.925138 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.925451 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.925734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] PATH_INFO: `/v3/users/109962dc7ec943b9aebc66d7ce6a26a5` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.926940 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109962dc7ec943b9aebc66d7ce6a26a5)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.929440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:54.064704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109962dc7ec943b9aebc66d7ce6a26a5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.069284 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:54.073171 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109962dc7ec943b9aebc66d7ce6a26a5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.073976 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109962dc7ec943b9aebc66d7ce6a26a5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.074660 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109962dc7ec943b9aebc66d7ce6a26a5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.087479 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109962dc7ec943b9aebc66d7ce6a26a5'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.176932 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1421/2833] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:53 2022] DELETE /identity/v3/users/109962dc7ec943b9aebc66d7ce6a26a5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:54.182765 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:54.183319 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:54.183564 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:54.202199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:54.203609 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:54.265769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1413/2834] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:54.275574 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:54.276987 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:54.336419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167723e8-3c91-4dce-bdd4-f2c998ad3e06 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:54.340160 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:54.340843 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:54.341141 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:54.341444 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:54.351546 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:54.353014 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:54.362581 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1422/2835] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:44:54.800679 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:54.802547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:54.815445 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94895f39-e665-425a-bc78-fdedf88e8fcc None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:54.820246 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:54.821150 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:54.821446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:54.821727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] PATH_INFO: `/v3/projects/aeb5c7b361be4ee98c893f096b150b85` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:54.822962 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb5c7b361be4ee98c893f096b150b85)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:54.827383 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:54.838923 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:54.954100 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb5c7b361be4ee98c893f096b150b85', 'request_id': 'req-4f59c023-1b4e-4723-b3d5-25287d95a158'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.959581 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb5c7b361be4ee98c893f096b150b85', 'request_id': 'req-4f59c023-1b4e-4723-b3d5-25287d95a158'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:54.967074 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb5c7b361be4ee98c893f096b150b85', 'request_id': 'req-4f59c023-1b4e-4723-b3d5-25287d95a158'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.056444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] The token cache is being invalidate because project aeb5c7b361be4ee98c893f096b150b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:55.056740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.057406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.058959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1414/2836] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:54 2022] DELETE /identity/v3/projects/aeb5c7b361be4ee98c893f096b150b85 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:55.062993 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.074593 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88192a4-abab-4e52-8cd9-730a685569df service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:55.135022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:55.135848 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.136140 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.136439 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] PATH_INFO: `/v3/users/5acb672014034061ac9864a1445fe2ee` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.137630 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5acb672014034061ac9864a1445fe2ee)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:55.139618 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:55.329190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5acb672014034061ac9864a1445fe2ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.336891 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5acb672014034061ac9864a1445fe2ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.337559 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5acb672014034061ac9864a1445fe2ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.338157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5acb672014034061ac9864a1445fe2ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.350729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-031860ea-62b0-4a73-b2b0-4bbb527ffbf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5acb672014034061ac9864a1445fe2ee'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:55.352510 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:55.467550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1423/2837] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:44:55 2022] DELETE /identity/v3/users/5acb672014034061ac9864a1445fe2ee => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:55.931628 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:55.933089 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:55.948096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59c023-1b4e-4723-b3d5-25287d95a158 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.010444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.011255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.011544 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.011822 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] PATH_INFO: `/v3/projects/bbf2e72f01384082921895e5615a3737` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.012987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf2e72f01384082921895e5615a3737)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.014908 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.124532 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf2e72f01384082921895e5615a3737', 'request_id': 'req-48b317c5-4855-498e-beee-966bb41fa505'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.130006 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf2e72f01384082921895e5615a3737', 'request_id': 'req-48b317c5-4855-498e-beee-966bb41fa505'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.136195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf2e72f01384082921895e5615a3737', 'request_id': 'req-48b317c5-4855-498e-beee-966bb41fa505'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.203929 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] The token cache is being invalidate because project bbf2e72f01384082921895e5615a3737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80342) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:44:56.204266 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.204883 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-48b317c5-4855-498e-beee-966bb41fa505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:44:56.206420 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1415/2838] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:44:55 2022] DELETE /identity/v3/projects/bbf2e72f01384082921895e5615a3737 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:56.212062 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed28bb2-333e-4623-92ea-53e00283b8d5 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.212533 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed28bb2-333e-4623-92ea-53e00283b8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.212627 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed28bb2-333e-4623-92ea-53e00283b8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.231222 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1ed28bb2-333e-4623-92ea-53e00283b8d5 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:56.244540 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1ed28bb2-333e-4623-92ea-53e00283b8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.295045 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1424/2839] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:56.305392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888a474-ac33-47e4-904a-1d59e019cb24 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.305392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888a474-ac33-47e4-904a-1d59e019cb24 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.305392 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888a474-ac33-47e4-904a-1d59e019cb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.323265 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-6888a474-ac33-47e4-904a-1d59e019cb24 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:56.324513 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-6888a474-ac33-47e4-904a-1d59e019cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.332428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1416/2840] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:56.337734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed28bb2-333e-4623-92ea-53e00283b8d5 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.347487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed28bb2-333e-4623-92ea-53e00283b8d5 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.352961 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.353616 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.353907 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.354190 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] PATH_INFO: `/v3/domains` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.354925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80343) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:56.355279 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.356911 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.368652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1425/2841] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:56.372760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888a474-ac33-47e4-904a-1d59e019cb24 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.384035 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888a474-ac33-47e4-904a-1d59e019cb24 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.388714 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.389406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.389692 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.389972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.390654 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.392595 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.457036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1417/2842] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:56.461966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.472036 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d07559-2909-4883-ad9b-278cc0bb7dac admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.528320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.528920 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.529214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.529500 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.530197 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.532159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.621776 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1426/2843] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:56.625847 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.631327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:56.636991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba86c1c-7025-470d-bb75-212e6190e2f8 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.641568 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.642208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.642494 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.642772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.643526 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.645111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.656041 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1418/2844] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:56.659930 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.670355 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ad432-8864-4f03-b471-1200af522e82 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.674995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ef769-8056-417a-9ffb-39351a5c4cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.675706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ef769-8056-417a-9ffb-39351a5c4cce admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.675992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ef769-8056-417a-9ffb-39351a5c4cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.676270 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ef769-8056-417a-9ffb-39351a5c4cce admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.677029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ef769-8056-417a-9ffb-39351a5c4cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.678675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ef769-8056-417a-9ffb-39351a5c4cce admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.688702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1427/2845] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:56.692769 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.703136 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74a2c5-8afd-4a29-b4ed-06eab30c9113 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.707941 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.708818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.709102 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.709409 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] PATH_INFO: `/v3/projects/bf954cda3da54036a341e88ef0db8251/users/6374aca515d1421b8d87c41afd76a113/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.726920 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf954cda3da54036a341e88ef0db8251, user_id=6374aca515d1421b8d87c41afd76a113, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.729747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.840291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1419/2846] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:56 2022] PUT /identity/v3/projects/bf954cda3da54036a341e88ef0db8251/users/6374aca515d1421b8d87c41afd76a113/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:56.850434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3651-21d4-44d5-8df3-d024062a9999 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.850736 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3651-21d4-44d5-8df3-d024062a9999 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.851022 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3651-21d4-44d5-8df3-d024062a9999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.878121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3a1d3651-21d4-44d5-8df3-d024062a9999 None None] MFA Rules not processed for user `6374aca515d1421b8d87c41afd76a113`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:56.879400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3a1d3651-21d4-44d5-8df3-d024062a9999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.924412 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:56.944320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1428/2847] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:56.949415 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.011571 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f47bf-32db-417e-985b-838682a43ea9 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.016380 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.017036 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.017326 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.017604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] PATH_INFO: `/v3/projects` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.018298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.020283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.108359 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1420/2848] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:44:56 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:57.112347 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3651-21d4-44d5-8df3-d024062a9999 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.123775 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3651-21d4-44d5-8df3-d024062a9999 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.178276 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.179545 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.179955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.180239 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] PATH_INFO: `/v3/users` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.180925 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.182845 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.261261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1429/2849] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:44:57.265190 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.275615 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57280e21-07d9-4936-bc74-12fbb0aeb5af admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.280287 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.280950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.281240 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.281524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.282255 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.283954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.295164 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1421/2850] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:57.299049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.310290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3d0f0-3146-433b-8c79-7a695d06ba5a admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.315230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.316121 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.316408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.316688 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] PATH_INFO: `/v3/projects/5e50e48c6d104bf48c0142c2d448c311/users/6a02348c278d4608b9492e968f685d52/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.331789 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e50e48c6d104bf48c0142c2d448c311, user_id=6a02348c278d4608b9492e968f685d52, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.334458 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.404586 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1430/2851] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:57 2022] PUT /identity/v3/projects/5e50e48c6d104bf48c0142c2d448c311/users/6a02348c278d4608b9492e968f685d52/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:57.408472 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.419249 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48555a-eb51-4949-8e0f-ef7dac81c384 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.474180 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ae6c3-91f8-465a-af71-1f08599c90d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.474842 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ae6c3-91f8-465a-af71-1f08599c90d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.475133 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ae6c3-91f8-465a-af71-1f08599c90d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.475447 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ae6c3-91f8-465a-af71-1f08599c90d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.476685 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ae6c3-91f8-465a-af71-1f08599c90d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.478274 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ae6c3-91f8-465a-af71-1f08599c90d3 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.487735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1422/2852] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:44:57.492408 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.502737 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22a1de-b486-48f0-9e07-b2abd6c42765 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.507652 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.508550 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.508839 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.509118 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] PATH_INFO: `/v3/projects/5e50e48c6d104bf48c0142c2d448c311/users/6a02348c278d4608b9492e968f685d52/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.511122 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e50e48c6d104bf48c0142c2d448c311, user_id=6a02348c278d4608b9492e968f685d52, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.513790 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.524228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1431/2853] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:44:57 2022] PUT /identity/v3/projects/5e50e48c6d104bf48c0142c2d448c311/users/6a02348c278d4608b9492e968f685d52/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:44:57.534195 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff967aa7-6f05-4c9a-92bc-22557dc3214a None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.534495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff967aa7-6f05-4c9a-92bc-22557dc3214a None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.534781 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff967aa7-6f05-4c9a-92bc-22557dc3214a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.565202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-ff967aa7-6f05-4c9a-92bc-22557dc3214a None None] MFA Rules not processed for user `6a02348c278d4608b9492e968f685d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:57.570734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-ff967aa7-6f05-4c9a-92bc-22557dc3214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.592719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.634951 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1423/2854] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:44:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:44:57.645024 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.646706 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:57.704739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f585c0-7364-475d-9b42-c072634e66fe admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.708311 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7febb-e338-4610-a1b8-4610cf6852f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.708955 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7febb-e338-4610-a1b8-4610cf6852f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.709247 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7febb-e338-4610-a1b8-4610cf6852f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.709529 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7febb-e338-4610-a1b8-4610cf6852f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.764093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.767739 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7febb-e338-4610-a1b8-4610cf6852f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.769180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7febb-e338-4610-a1b8-4610cf6852f6 service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.778087 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1432/2855] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:00.756446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff967aa7-6f05-4c9a-92bc-22557dc3214a None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:00.817345 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff967aa7-6f05-4c9a-92bc-22557dc3214a None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:00.822411 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:00.823305 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:00.823592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:00.823983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] PATH_INFO: `/v3/users/6374aca515d1421b8d87c41afd76a113` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:00.825165 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6374aca515d1421b8d87c41afd76a113)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:00.827072 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.024401 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6374aca515d1421b8d87c41afd76a113'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.025028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6374aca515d1421b8d87c41afd76a113'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.031790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6374aca515d1421b8d87c41afd76a113'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.032422 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6374aca515d1421b8d87c41afd76a113'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.107202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6374aca515d1421b8d87c41afd76a113'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.125557 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1424/2856] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:00 2022] DELETE /identity/v3/users/6374aca515d1421b8d87c41afd76a113 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:01.130183 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.130657 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.130768 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.148143 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:01.149562 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.210728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1433/2857] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:01.220734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.222184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.268961 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:01.280329 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfe313-8311-4060-b339-10124ca4ca74 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.283930 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.284573 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.284858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.285132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.296037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.297465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.307650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1425/2858] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:01.754820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.756290 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.769259 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2834e15-2e54-42e6-a408-41b75d39978f None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.774250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.775859 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.777598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.777598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] PATH_INFO: `/v3/projects/bf954cda3da54036a341e88ef0db8251` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.777598 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf954cda3da54036a341e88ef0db8251)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.779576 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.903569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf954cda3da54036a341e88ef0db8251', 'request_id': 'req-795f27e0-8cfa-4e59-8966-6ed77a476b50'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.909771 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf954cda3da54036a341e88ef0db8251', 'request_id': 'req-795f27e0-8cfa-4e59-8966-6ed77a476b50'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.916148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf954cda3da54036a341e88ef0db8251', 'request_id': 'req-795f27e0-8cfa-4e59-8966-6ed77a476b50'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.986235 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] The token cache is being invalidate because project bf954cda3da54036a341e88ef0db8251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:01.986536 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.987168 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:01.988738 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1434/2859] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:01 2022] DELETE /identity/v3/projects/bf954cda3da54036a341e88ef0db8251 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:01.992687 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.002690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c35ff9-6885-456e-afff-139d32c72209 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.063080 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.064046 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.064298 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.064579 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] PATH_INFO: `/v3/users/6a02348c278d4608b9492e968f685d52` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.065853 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a02348c278d4608b9492e968f685d52)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.067799 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.309004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a02348c278d4608b9492e968f685d52'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.309747 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a02348c278d4608b9492e968f685d52'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.317303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a02348c278d4608b9492e968f685d52'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.318007 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a02348c278d4608b9492e968f685d52'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.420757 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a02348c278d4608b9492e968f685d52'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:02.439105 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1426/2860] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:01 2022] DELETE /identity/v3/users/6a02348c278d4608b9492e968f685d52 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:02.924228 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.925682 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:02.939659 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f27e0-8cfa-4e59-8966-6ed77a476b50 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.997725 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.998587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.998876 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.999158 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] PATH_INFO: `/v3/projects/5e50e48c6d104bf48c0142c2d448c311` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.000376 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e50e48c6d104bf48c0142c2d448c311)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.002293 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.023265 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:03.135779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e50e48c6d104bf48c0142c2d448c311', 'request_id': 'req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.141223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e50e48c6d104bf48c0142c2d448c311', 'request_id': 'req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.147608 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e50e48c6d104bf48c0142c2d448c311', 'request_id': 'req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.226878 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] The token cache is being invalidate because project 5e50e48c6d104bf48c0142c2d448c311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:03.227252 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.231418 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d0b479b0-98fa-4861-a87f-85d6e8eba4c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:03.237645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1435/2861] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:02 2022] DELETE /identity/v3/projects/5e50e48c6d104bf48c0142c2d448c311 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:07.995169 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:07.996717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:08.009874 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33c986-af42-41c8-bdf3-60892526a29e admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:08.067841 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:08.068725 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:08.069008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:08.069289 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] PATH_INFO: `/v3/users/55b4ee7185a2437aaf5e29ff2cfce622` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:08.070495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b4ee7185a2437aaf5e29ff2cfce622)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:08.072556 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:08.077140 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:08.234960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b4ee7185a2437aaf5e29ff2cfce622'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:08.235561 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b4ee7185a2437aaf5e29ff2cfce622'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:08.242501 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b4ee7185a2437aaf5e29ff2cfce622'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:08.243132 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b4ee7185a2437aaf5e29ff2cfce622'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:08.329728 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b4ee7185a2437aaf5e29ff2cfce622'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:08.348464 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1427/2862] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:07 2022] DELETE /identity/v3/users/55b4ee7185a2437aaf5e29ff2cfce622 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:08.354169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:08.354169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:08.354786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:08.371568 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:08.372774 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:08.430261 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1436/2863] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:08.439905 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:08.441296 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:08.500232 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f27b2-ad83-43c9-816f-1e86a729ee0c admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:08.503986 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155fdda-fbae-41e5-8f24-f90cc6114164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:08.504638 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155fdda-fbae-41e5-8f24-f90cc6114164 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:08.504928 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155fdda-fbae-41e5-8f24-f90cc6114164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:08.505208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155fdda-fbae-41e5-8f24-f90cc6114164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:08.514850 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155fdda-fbae-41e5-8f24-f90cc6114164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:08.516318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155fdda-fbae-41e5-8f24-f90cc6114164 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:08.526419 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1428/2864] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:08.920250 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:08.921779 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:08.934596 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516cd43-1640-4379-9f8b-b631846d0f22 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:08.939560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:08.940464 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:08.940750 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:08.941029 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] PATH_INFO: `/v3/projects/a9753aef45e649ac9fa4a0736ab43b7c` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:08.942233 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9753aef45e649ac9fa4a0736ab43b7c)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:08.944205 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.062136 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9753aef45e649ac9fa4a0736ab43b7c', 'request_id': 'req-d52b9eb7-10a3-4c3f-87e8-510d2015c471'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:09.067462 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9753aef45e649ac9fa4a0736ab43b7c', 'request_id': 'req-d52b9eb7-10a3-4c3f-87e8-510d2015c471'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:09.074414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9753aef45e649ac9fa4a0736ab43b7c', 'request_id': 'req-d52b9eb7-10a3-4c3f-87e8-510d2015c471'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:09.151901 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] The token cache is being invalidate because project a9753aef45e649ac9fa4a0736ab43b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:09.152193 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:09.152812 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-d52b9eb7-10a3-4c3f-87e8-510d2015c471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:09.154374 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1437/2865] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:08 2022] DELETE /identity/v3/projects/a9753aef45e649ac9fa4a0736ab43b7c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:09.159239 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710adee-d365-404a-bab1-ab72791528fd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.162740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710adee-d365-404a-bab1-ab72791528fd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.162740 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710adee-d365-404a-bab1-ab72791528fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.177972 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9710adee-d365-404a-bab1-ab72791528fd None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:09.179202 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9710adee-d365-404a-bab1-ab72791528fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:09.237903 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1429/2866] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:09.246356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b491dff-2dcc-4fb2-801c-28bb8baf7fa4 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.246649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b491dff-2dcc-4fb2-801c-28bb8baf7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.246933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b491dff-2dcc-4fb2-801c-28bb8baf7fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.265042 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-1b491dff-2dcc-4fb2-801c-28bb8baf7fa4 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:09.266249 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1b491dff-2dcc-4fb2-801c-28bb8baf7fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:09.274107 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1438/2867] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:09.278361 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710adee-d365-404a-bab1-ab72791528fd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:09.287950 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710adee-d365-404a-bab1-ab72791528fd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:09.293386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:09.294096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.294382 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.294666 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.295432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:09.295744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:09.297360 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.309586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1430/2868] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:09.313480 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b491dff-2dcc-4fb2-801c-28bb8baf7fa4 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:09.322819 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b491dff-2dcc-4fb2-801c-28bb8baf7fa4 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:09.327502 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:09.328150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.328440 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.328721 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.329421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:09.331349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.410988 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1439/2869] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:09 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:09.414790 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:09.425293 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d199dde-3fd6-4aed-b607-09027ac4bf27 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:09.435003 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:09.483291 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:09.483965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.484256 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.484540 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.485230 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:09.487146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.560159 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1431/2870] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:09 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:09.563697 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:09.569295 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:09.574625 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096b6a5-38e4-4654-b413-9063195bca79 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:09.579149 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:09.579830 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.580116 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.580400 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.581125 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:09.582716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.593829 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1440/2871] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:09.598547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:09.609470 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd7934-209e-478a-badf-58da7084e172 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:09.614312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32011a-5189-4a3a-8764-31c8a0f622eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:09.615042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32011a-5189-4a3a-8764-31c8a0f622eb admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.615363 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32011a-5189-4a3a-8764-31c8a0f622eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.615650 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32011a-5189-4a3a-8764-31c8a0f622eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.616372 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32011a-5189-4a3a-8764-31c8a0f622eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:09.617943 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32011a-5189-4a3a-8764-31c8a0f622eb admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.628468 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1432/2872] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:09.632377 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:09.642720 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c3ac4-05d1-431f-bf5f-e46b5ef685ce admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:09.647560 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:09.648434 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.648724 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.649008 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] PATH_INFO: `/v3/projects/543805dc2b6a43eb92b28d4bfeba7900/users/6f4cc07bcb9244fb832cd7ce1084d10b/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.665268 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=543805dc2b6a43eb92b28d4bfeba7900, user_id=6f4cc07bcb9244fb832cd7ce1084d10b, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:09.668701 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.747367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1441/2873] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:09 2022] PUT /identity/v3/projects/543805dc2b6a43eb92b28d4bfeba7900/users/6f4cc07bcb9244fb832cd7ce1084d10b/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:09.757560 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7f861-5e8c-4330-a62a-f8e21c3655bd None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.757870 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7f861-5e8c-4330-a62a-f8e21c3655bd None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.758156 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7f861-5e8c-4330-a62a-f8e21c3655bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.784907 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-85c7f861-5e8c-4330-a62a-f8e21c3655bd None None] MFA Rules not processed for user `6f4cc07bcb9244fb832cd7ce1084d10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:09.786162 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-85c7f861-5e8c-4330-a62a-f8e21c3655bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:09.849330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1433/2874] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:09.861273 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:09.862972 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:09.921587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc5c70-1b2d-4096-aaac-a7f0f9aa0a68 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:09.925230 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3391d11-82d3-440f-be32-1a4a3646795c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:09.925875 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3391d11-82d3-440f-be32-1a4a3646795c service cinder] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:09.926159 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3391d11-82d3-440f-be32-1a4a3646795c service cinder] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:09.926436 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3391d11-82d3-440f-be32-1a4a3646795c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:09.935840 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3391d11-82d3-440f-be32-1a4a3646795c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:09.937313 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3391d11-82d3-440f-be32-1a4a3646795c service cinder] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:09.946307 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1442/2875] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:19.850764 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:19.942220 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:21.127070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7f861-5e8c-4330-a62a-f8e21c3655bd None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:21.190658 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7f861-5e8c-4330-a62a-f8e21c3655bd None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:21.195062 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:21.195954 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:21.196250 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:21.196524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] PATH_INFO: `/v3/users/6f4cc07bcb9244fb832cd7ce1084d10b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:21.198262 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4cc07bcb9244fb832cd7ce1084d10b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:21.200461 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:21.405199 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4cc07bcb9244fb832cd7ce1084d10b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.405858 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4cc07bcb9244fb832cd7ce1084d10b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.412874 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4cc07bcb9244fb832cd7ce1084d10b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.413499 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4cc07bcb9244fb832cd7ce1084d10b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.487665 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4cc07bcb9244fb832cd7ce1084d10b'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:21.505096 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1434/2876] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:45:21 2022] DELETE /identity/v3/users/6f4cc07bcb9244fb832cd7ce1084d10b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:21.509609 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:21.510595 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:21.510898 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:21.529090 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:21.530731 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:21.590919 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1443/2877] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:21.605531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:21.606991 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:21.664297 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06007543-1d7f-478e-be61-72f994fc5b02 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:21.667957 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793cfb56-5a9a-4ac1-8bcb-aac553d9314f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:21.668592 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cfb56-5a9a-4ac1-8bcb-aac553d9314f service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:21.668875 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cfb56-5a9a-4ac1-8bcb-aac553d9314f service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:21.669154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cfb56-5a9a-4ac1-8bcb-aac553d9314f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:21.677952 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793cfb56-5a9a-4ac1-8bcb-aac553d9314f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:21.679413 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793cfb56-5a9a-4ac1-8bcb-aac553d9314f service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:21.691431 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1435/2878] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:22.056275 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.058351 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:22.071995 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbecac5-6485-4c0c-91b7-a9bce102035d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.077202 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.078066 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.078349 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.078617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] PATH_INFO: `/v3/projects/543805dc2b6a43eb92b28d4bfeba7900` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.079860 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] RBAC: Authorizing `identity:delete_project(project_id=543805dc2b6a43eb92b28d4bfeba7900)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.081744 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.193515 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543805dc2b6a43eb92b28d4bfeba7900', 'request_id': 'req-bdae8a89-0fee-49a6-9836-8d006f50256b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.199337 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543805dc2b6a43eb92b28d4bfeba7900', 'request_id': 'req-bdae8a89-0fee-49a6-9836-8d006f50256b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.205933 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543805dc2b6a43eb92b28d4bfeba7900', 'request_id': 'req-bdae8a89-0fee-49a6-9836-8d006f50256b'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.300339 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] The token cache is being invalidate because project 543805dc2b6a43eb92b28d4bfeba7900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:45:22.300558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.301174 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-bdae8a89-0fee-49a6-9836-8d006f50256b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:45:22.302743 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1444/2879] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:45:22 2022] DELETE /identity/v3/projects/543805dc2b6a43eb92b28d4bfeba7900 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:22.308146 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e01401-152b-4091-80b9-8949b3a718d7 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.308633 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e01401-152b-4091-80b9-8949b3a718d7 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.308745 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e01401-152b-4091-80b9-8949b3a718d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.327835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-d9e01401-152b-4091-80b9-8949b3a718d7 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:22.329058 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d9e01401-152b-4091-80b9-8949b3a718d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:22.346835 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:22.389013 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1436/2880] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:22.397700 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a580-1450-4a16-bcb9-956e3c70c60d None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.397992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a580-1450-4a16-bcb9-956e3c70c60d None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.398277 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a580-1450-4a16-bcb9-956e3c70c60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.418383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e883a580-1450-4a16-bcb9-956e3c70c60d None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:22.419645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e883a580-1450-4a16-bcb9-956e3c70c60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:22.427702 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1445/2881] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:22.431818 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e01401-152b-4091-80b9-8949b3a718d7 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.441547 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e01401-152b-4091-80b9-8949b3a718d7 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.446303 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.446956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.447366 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.447777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.448516 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80342) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:22.448829 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.450424 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.462145 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1437/2882] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 11:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:22.466942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a580-1450-4a16-bcb9-956e3c70c60d None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.476320 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a580-1450-4a16-bcb9-956e3c70c60d None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.481655 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.482309 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.482649 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.482936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.483668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.485716 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.629243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1446/2883] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:22 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:22.632808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.643589 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e02ad0-2191-44ed-9b22-536647fe75d5 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.699644 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.700327 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.700616 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.700901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.701604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.703552 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.827784 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1438/2884] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:22 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:22.831636 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.843416 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f71e4-a7d8-4257-b464-7c1334da62d6 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.848547 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.849213 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.849501 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.849780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.850513 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.852148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.862758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1447/2885] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:22.866752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.878038 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda12715-93f9-44fd-a924-7246ca64a69b admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.882771 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f621f-705a-4d35-a09a-4e739cac2416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.883406 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f621f-705a-4d35-a09a-4e739cac2416 admin admin] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.883698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f621f-705a-4d35-a09a-4e739cac2416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.883983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f621f-705a-4d35-a09a-4e739cac2416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.885225 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f621f-705a-4d35-a09a-4e739cac2416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.886823 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f621f-705a-4d35-a09a-4e739cac2416 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.896760 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1439/2886] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:22.901486 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.913243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db2a8c-26fe-475b-acad-3f2c589bbfce admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.917034 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.917924 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.918602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.918602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] PATH_INFO: `/v3/projects/8ae8ba9675094e419578fd820d926509/users/6e7ef942117844d281d4b4c1d4fd20b5/roles/adb9ae992e5a4594a8f7510556b4db5f` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.932971 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae8ba9675094e419578fd820d926509, user_id=6e7ef942117844d281d4b4c1d4fd20b5, role_id=adb9ae992e5a4594a8f7510556b4db5f)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.935699 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:22.986634 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1448/2887] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:22 2022] PUT /identity/v3/projects/8ae8ba9675094e419578fd820d926509/users/6e7ef942117844d281d4b4c1d4fd20b5/roles/adb9ae992e5a4594a8f7510556b4db5f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:22.991690 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa017-3413-4c29-8d23-42fb69fea003 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.991987 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa017-3413-4c29-8d23-42fb69fea003 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.992271 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fa017-3413-4c29-8d23-42fb69fea003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:23.009619 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-587fa017-3413-4c29-8d23-42fb69fea003 None None] MFA Rules not processed for user `912834523da04e79b38964af43312d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:23.011442 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-587fa017-3413-4c29-8d23-42fb69fea003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:23.070269 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1440/2888] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:23.080299 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:23.082049 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:23.103445 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:23.139211 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf2353-8220-462c-aaf3-95206e36da72 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:23.142846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:23.143524 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:23.143815 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:23.144097 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:23.153710 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:23.155166 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:23.164602 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1449/2889] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:28.393705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b79f3-8c2c-46e1-aa87-4b3095120711 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.394222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b79f3-8c2c-46e1-aa87-4b3095120711 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.394330 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b79f3-8c2c-46e1-aa87-4b3095120711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.422502 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-9c0b79f3-8c2c-46e1-aa87-4b3095120711 None None] MFA Rules not processed for user `6e7ef942117844d281d4b4c1d4fd20b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:28.423815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-9c0b79f3-8c2c-46e1-aa87-4b3095120711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:28.442185 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:28.497099 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1441/2890] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:45:28.501758 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.513223 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b3ff7-3368-4179-93f1-0ce767fb9f02 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.518206 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.518882 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.519169 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.519474 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] PATH_INFO: `/v3/projects` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.520182 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.522242 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.624258 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1450/2891] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:28.628154 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b79f3-8c2c-46e1-aa87-4b3095120711 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.640459 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b79f3-8c2c-46e1-aa87-4b3095120711 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.835349 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.836037 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.836325 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.836604 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] PATH_INFO: `/v3/users` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.837311 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.839776 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.947368 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1442/2892] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 11:45:28 2022] POST /identity/v3/users => generated 292 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:28.951380 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.962093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee74fb-c3e1-4af2-950e-7410dcce1fab admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.967170 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.967866 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.968155 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.968438 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.969184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.970780 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.982360 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1451/2893] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:28.986480 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.997028 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acc189-2800-43ca-a083-82e973587477 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.001618 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.002488 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.002772 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.003049 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] PATH_INFO: `/v3/projects/efac6e4471124ebb990bca8da0621c7d/users/71bdd0a80a2f418a8a7bcf8c3b0912ca/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.017684 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=efac6e4471124ebb990bca8da0621c7d, user_id=71bdd0a80a2f418a8a7bcf8c3b0912ca, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.020531 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.084497 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1443/2894] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:28 2022] PUT /identity/v3/projects/efac6e4471124ebb990bca8da0621c7d/users/71bdd0a80a2f418a8a7bcf8c3b0912ca/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:29.088872 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.098800 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b550183-6431-4fe5-80c7-d7f433603a18 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.154164 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97052991-e8d0-48b0-a328-c353f4dd5e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.154827 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052991-e8d0-48b0-a328-c353f4dd5e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.155113 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052991-e8d0-48b0-a328-c353f4dd5e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.155421 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97052991-e8d0-48b0-a328-c353f4dd5e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.156148 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97052991-e8d0-48b0-a328-c353f4dd5e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.157788 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97052991-e8d0-48b0-a328-c353f4dd5e0f admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.168084 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1452/2895] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 11:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:29.172208 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:29.183247 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bf231-7484-496f-a86e-6c6d4c6424fe admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:29.187839 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:29.188717 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:29.189004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:29.189288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] PATH_INFO: `/v3/projects/efac6e4471124ebb990bca8da0621c7d/users/71bdd0a80a2f418a8a7bcf8c3b0912ca/roles/dd9dd4b6e3c14eb4804574a721ad656b` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:29.191288 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efac6e4471124ebb990bca8da0621c7d, user_id=71bdd0a80a2f418a8a7bcf8c3b0912ca, role_id=dd9dd4b6e3c14eb4804574a721ad656b)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:29.193955 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:29.205200 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1444/2896] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 11:45:29 2022] PUT /identity/v3/projects/efac6e4471124ebb990bca8da0621c7d/users/71bdd0a80a2f418a8a7bcf8c3b0912ca/roles/dd9dd4b6e3c14eb4804574a721ad656b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:34.831666 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04710f0-17a5-4761-ae85-e9db568c7b00 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:34.832072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04710f0-17a5-4761-ae85-e9db568c7b00 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:34.832150 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04710f0-17a5-4761-ae85-e9db568c7b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:34.862385 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-e04710f0-17a5-4761-ae85-e9db568c7b00 None None] MFA Rules not processed for user `71bdd0a80a2f418a8a7bcf8c3b0912ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:34.863601 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e04710f0-17a5-4761-ae85-e9db568c7b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:34.966520 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1453/2897] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:34.983888 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:34.985689 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:35.064465 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:35.070946 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87006f0e-22a7-4479-935b-4bc6e4fab946 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:35.076786 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06802-8038-486d-98ba-42983916a82b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:35.077444 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06802-8038-486d-98ba-42983916a82b service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:35.077729 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06802-8038-486d-98ba-42983916a82b service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:35.078008 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06802-8038-486d-98ba-42983916a82b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:35.137111 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06802-8038-486d-98ba-42983916a82b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:35.138727 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06802-8038-486d-98ba-42983916a82b service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:35.148698 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1445/2898] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:37.839076 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:37.839663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:37.839663 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:37.858881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] MFA Rules not processed for user `6e7ef942117844d281d4b4c1d4fd20b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:37.860184 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:37.867667 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1454/2899] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 11:45:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:45:45.144648 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=27 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:47.852820 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:54.433524 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06802-8038-486d-98ba-42983916a82b service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:54.435085 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-39c06802-8038-486d-98ba-42983916a82b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:54.492222 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06802-8038-486d-98ba-42983916a82b service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:54.495752 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2678deaa97fc4012aa0d71f9e505f5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:54.496386 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:54.496670 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:54.496992 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:54.498861 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:54.511845 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:54.513304 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:54.533520 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1446/2900] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:54.808504 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:54.809936 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:54.867225 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06e752-0d80-4a91-bce7-e91ca53658e3 None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:54.870723 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:54.871397 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:54.871693 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:54.871978 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:54.873746 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:54.883356 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:54.884825 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:54.893435 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1455/2901] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:55.667110 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:55.669073 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:55.739584 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c92969-06c5-48db-b8ec-eed964796bde service cinder] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:55.744495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:55.745141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:55.745428 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:55.745705 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:55.748141 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:55.759394 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:55.760960 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:55.770792 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1447/2902] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:58.839147 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:58.849496 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97628733-51c3-44c1-ad44-1a7cbba0cb15 service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:58.853392 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:58.854364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:58.854729 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:58.855085 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:58.857583 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:58.869030 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:58.872556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:58.882330 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1456/2903] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:45:59.720281 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:59.737004 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1db9c1-b6f8-4b57-9038-bfd56c3159ff service placement] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:59.743548 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:59.743831 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:59.744114 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:59.744389 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:59.746479 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:59.759314 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:59.760899 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:59.772704 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1448/2904] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:45:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:08.877571 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=32 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:09.767423 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=68 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:12.492264 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:12.506180 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0cf6b-6e00-40cb-bf13-1bb64efb5d3b service neutron] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:12.512405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8ab7c-a467-4570-a977-44e405d69e78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:12.513214 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8ab7c-a467-4570-a977-44e405d69e78 service nova] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:12.513505 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8ab7c-a467-4570-a977-44e405d69e78 service nova] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:12.513786 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8ab7c-a467-4570-a977-44e405d69e78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:12.515569 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-adc8ab7c-a467-4570-a977-44e405d69e78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:12.527837 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8ab7c-a467-4570-a977-44e405d69e78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:12.529419 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8ab7c-a467-4570-a977-44e405d69e78 service nova] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:12.540302 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1457/2905] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:14.572612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:14.582336 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742fdb-5b79-46d4-8dc6-9781f2dc710a service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:14.585805 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195d794-d41b-4d19-b386-667d7171450e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:14.586432 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195d794-d41b-4d19-b386-667d7171450e service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:14.586719 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195d794-d41b-4d19-b386-667d7171450e service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:14.586995 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195d794-d41b-4d19-b386-667d7171450e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:14.595777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195d794-d41b-4d19-b386-667d7171450e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:14.597184 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195d794-d41b-4d19-b386-667d7171450e service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:14.607044 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1449/2906] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:22.535604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:24.602284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=6 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:26.532461 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e87f19-247e-400e-bfe7-e9c9347189db None None] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:26.534881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e87f19-247e-400e-bfe7-e9c9347189db None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:26.534881 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e87f19-247e-400e-bfe7-e9c9347189db None None] PATH_INFO: `/` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.535207 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1458/2907] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:48:26.542339 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1278e-12ad-42ab-9617-b7a8373353b5 None None] REQUEST_METHOD: `POST` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:26.542959 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1278e-12ad-42ab-9617-b7a8373353b5 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:26.543590 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1278e-12ad-42ab-9617-b7a8373353b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.595528 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.auth.core [None req-91c1278e-12ad-42ab-9617-b7a8373353b5 None None] MFA Rules not processed for user `cf08ecfc3fed44e2bc1ccff8521a4ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80342) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:26.597653 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-91c1278e-12ad-42ab-9617-b7a8373353b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:26.604284 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1450/2908] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:48:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:26.617704 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e87f19-247e-400e-bfe7-e9c9347189db None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:26.629072 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e87f19-247e-400e-bfe7-e9c9347189db None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:26.632734 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba8ed3-86e8-42d4-8bb6-61c4b2278e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:26.633373 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba8ed3-86e8-42d4-8bb6-61c4b2278e27 service neutron] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:26.633675 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba8ed3-86e8-42d4-8bb6-61c4b2278e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:26.633942 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba8ed3-86e8-42d4-8bb6-61c4b2278e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.641132 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba8ed3-86e8-42d4-8bb6-61c4b2278e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:26.642623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba8ed3-86e8-42d4-8bb6-61c4b2278e27 service neutron] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:26.650994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1459/2909] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:26.768123 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245fe84-a2f5-49d5-ab13-e786c9b8f255 None None] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:26.768956 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245fe84-a2f5-49d5-ab13-e786c9b8f255 None None] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:26.769226 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245fe84-a2f5-49d5-ab13-e786c9b8f255 None None] PATH_INFO: `/` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.770586 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1451/2910] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 11:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:26.775998 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] REQUEST_METHOD: `POST` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:26.776319 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:26.776604 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.813367 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.auth.core [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] MFA Rules not processed for user `54bb42878fff477099bbf56a46e9eec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80343) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:26.814623 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:26.824994 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1460/2911] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 11:48:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:26.836192 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245fe84-a2f5-49d5-ab13-e786c9b8f255 None None] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:26.847384 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245fe84-a2f5-49d5-ab13-e786c9b8f255 None None] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:26.850808 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf08ecfc3fed44e2bc1ccff8521a4ed8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:26.851473 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:26.851763 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:26.852042 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.862090 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:26.863601 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:26.875439 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1452/2912] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:48:36.810966 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=15 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:36.870640 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=19 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:48.545617 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:48.546999 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:48.563489 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2552fd-8cb4-4176-b171-eabb2841cdad None None] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:48.568567 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:48.569237 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:48.569526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:48.569804 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:48.571656 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:48.583846 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:48.585414 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:48.595243 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1461/2913] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:58.590992 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=6 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:55.720744 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:55.734224 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6633df-a658-49c3-8c56-76b7204f94e7 service neutron] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:55.741931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bb42878fff477099bbf56a46e9eec0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:55.742612 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:55.742868 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:55.743144 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:55.747306 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:55.770312 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:55.772508 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:55.783350 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1453/2914] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:03.956405 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:03.970470 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb346b2-de2f-4bba-8370-4d32cba11b01 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:03.979154 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:03.990852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] REQUEST_METHOD: `GET` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:03.990852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:03.990852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:03.990852 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:03.996051 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:03.997728 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:04.008668 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1462/2915] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:05.778866 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=11 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:07.673400 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:07.755029 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb6e2-1038-4cb1-b6c1-33e1bd002801 service nova] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:07.760237 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:07.761092 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:07.761373 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:07.761649 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] PATH_INFO: `/v3/users/6e7ef942117844d281d4b4c1d4fd20b5` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:07.762815 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7ef942117844d281d4b4c1d4fd20b5)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:07.764775 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:07.945070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7ef942117844d281d4b4c1d4fd20b5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:07.945070 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7ef942117844d281d4b4c1d4fd20b5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:07.946975 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7ef942117844d281d4b4c1d4fd20b5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:07.947643 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7ef942117844d281d4b4c1d4fd20b5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.033283 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7ef942117844d281d4b4c1d4fd20b5'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.052158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1454/2916] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:50:07 2022] DELETE /identity/v3/users/6e7ef942117844d281d4b4c1d4fd20b5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:08.290794 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:08.301836 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7b3e2-655d-4e7d-a1d6-a628ba7c5a07 service placement] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:08.362383 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:08.363296 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:08.363587 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:08.363868 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] PATH_INFO: `/v3/projects/8ae8ba9675094e419578fd820d926509` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:08.365673 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae8ba9675094e419578fd820d926509)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:08.367632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:08.467632 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae8ba9675094e419578fd820d926509', 'request_id': 'req-e7dbb434-9eab-4dac-9a31-38f44d081376'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.473364 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae8ba9675094e419578fd820d926509', 'request_id': 'req-e7dbb434-9eab-4dac-9a31-38f44d081376'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.480511 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae8ba9675094e419578fd820d926509', 'request_id': 'req-e7dbb434-9eab-4dac-9a31-38f44d081376'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.581558 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] The token cache is being invalidate because project 8ae8ba9675094e419578fd820d926509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:50:08.581855 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.582487 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.584767 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1463/2917] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:50:08 2022] DELETE /identity/v3/projects/8ae8ba9675094e419578fd820d926509 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:08.588860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:08.590295 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:08.604734 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75175a-b673-41ba-93e5-bd2481eac48a admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:08.661475 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:08.663001 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:08.663318 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:08.663599 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] PATH_INFO: `/v3/users/71bdd0a80a2f418a8a7bcf8c3b0912ca` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:08.664846 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bdd0a80a2f418a8a7bcf8c3b0912ca)` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:08.666758 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:08.669158 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:08.909966 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bdd0a80a2f418a8a7bcf8c3b0912ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.910583 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bdd0a80a2f418a8a7bcf8c3b0912ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.918812 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bdd0a80a2f418a8a7bcf8c3b0912ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.919260 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bdd0a80a2f418a8a7bcf8c3b0912ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.980777 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.notifications [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bdd0a80a2f418a8a7bcf8c3b0912ca'} {{(pid=80342) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:08.998563 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1455/2918] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 11:50:08 2022] DELETE /identity/v3/users/71bdd0a80a2f418a8a7bcf8c3b0912ca => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:09.245535 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Authenticating user token {{(pid=80343) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:09.247001 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.fernet_utils [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80343) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:09.260588 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbb434-9eab-4dac-9a31-38f44d081376 admin admin] Validating token access rules against request {{(pid=80343) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:09.318549 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '912834523da04e79b38964af43312d61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd945ea14d2824c65a22104765fba4b15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80343) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:09.319863 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:09.320144 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:09.320417 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] PATH_INFO: `/v3/projects/efac6e4471124ebb990bca8da0621c7d` {{(pid=80343) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:09.321645 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efac6e4471124ebb990bca8da0621c7d)` {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:09.323592 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] RBAC: Authorization granted {{(pid=80343) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:09.331556 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:09.449760 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efac6e4471124ebb990bca8da0621c7d', 'request_id': 'req-7afdabbb-4cb1-4924-8b7f-4524284e5710'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:09.455093 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efac6e4471124ebb990bca8da0621c7d', 'request_id': 'req-7afdabbb-4cb1-4924-8b7f-4524284e5710'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:09.461208 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efac6e4471124ebb990bca8da0621c7d', 'request_id': 'req-7afdabbb-4cb1-4924-8b7f-4524284e5710'} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:09.546300 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] The token cache is being invalidate because project efac6e4471124ebb990bca8da0621c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80343) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:50:09.546526 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:09.547157 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG keystone.notifications [None req-7afdabbb-4cb1-4924-8b7f-4524284e5710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80343) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:09.548733 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: [pid: 80343|app: 0|req: 1464/2919] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 11:50:09 2022] DELETE /identity/v3/projects/efac6e4471124ebb990bca8da0621c7d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:18.994901 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:19.543021 dv-cinder-826607-6-iscsi devstack@keystone.service[80343]: DEBUG dbcounter [-] [80343] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80343) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:54.516965 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Authenticating user token {{(pid=80342) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:54.518446 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:54.607735 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9a385-6bb9-4786-b357-03b7af0f4c58 admin admin] Validating token access rules against request {{(pid=80342) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:54.664916 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05959ae6-992c-4e86-96d5-0d40cb43ef46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80003dcc49ef47c8a08887462d102b5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87b0060a49c54aa79beba6f2a83ea381', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80342) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:54.665575 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959ae6-992c-4e86-96d5-0d40cb43ef46 service placement] REQUEST_METHOD: `GET` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:54.665860 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959ae6-992c-4e86-96d5-0d40cb43ef46 service placement] SCRIPT_NAME: `/identity` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:54.666134 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959ae6-992c-4e86-96d5-0d40cb43ef46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80342) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:54.667931 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.fernet_utils [None req-05959ae6-992c-4e86-96d5-0d40cb43ef46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80342) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:54.679596 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05959ae6-992c-4e86-96d5-0d40cb43ef46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:54.681148 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05959ae6-992c-4e86-96d5-0d40cb43ef46 service placement] RBAC: Authorization granted {{(pid=80342) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:54.691495 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: [pid: 80342|app: 0|req: 1456/2920] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 11:53:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:04.686983 dv-cinder-826607-6-iscsi devstack@keystone.service[80342]: DEBUG dbcounter [-] [80342] Writing DB stats keystone:SELECT=67 {{(pid=80342) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}